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SOMMAKT 


OBJECTIVE 

Assess  the  accuracy  of  predicted  MJFs  by  prediction  programs,  commonly 
used  in  Navy  applications,  using  a  previously  assembled  data  base  of  observed 
oblique  sounder  median  MOFs  from  25  paths. 

RESULTS 

1.  The  programs  conpared  were  the  ITSA- 1 ,  HFMUFES  4,  MINIMUF-3.5,  and  the 
version  of  ITSA-1  used  to  produce  Naval  Telecommunications  Publication  (NTP)  6 
Supplement  1 . 

2.  Predicted  MUFs  for  these  25  paths  and  the  observed  MDFs  themselves  were 
screened  into  nine  subsets  of  data  to  see  the  effect  of  particular  paths,  path 
length,  path  orientation,  season,  month,  latitude,  solar  sunspot  number, 
diurnal  trends,  geographic  region  and  sounder  type. 

3.  An  indication  of  the  accuracy  of  the  numerical  predictions  of  MUF  was 
obtained  from  the  study  of  the  residuals  between  the  observed  data  and 
predicted  values. 

4.  MINIMUF  3.5  appeared  most  accurate  overall,  with  its  bias  of  0.08  MHz  (0.6 
percent)  and  an  rms  error  of  3.71  MHz  (3.6  percent). 

5.  MINIMUF  3.5  had  difficulty  predicting  accurately  during  the  sunrise  and 
sunset  transition  hours  and  for  path  lengths  5000  to  7000  km. 

6.  The  version  of  ITSA-1  with  ionospheric  characteristics  mapped  in  universal 
time  was  slightly  more  accurate  than  the  version  of  ITSA-1  used  to  produce  NTP 
6  Supp.  1 . 

7.  Except  for  land  paths,  the  performance  of  HFMUFES  4  was  disappointing;  it 
was  biased  1.49  MHz  high  (7.2  percent  high)  and  rms  error  of  4.24  MHz  (8.3 
percent) . 

8.  HFMJFES  4  had  difficulty  predicting  MUFs  accurately  for  paths  over  ocean 
areas  and  for  paths  with  lengths  between  4000  and  5000  km. 


9.  All  of  the  programs  had  difficulty  predicting  MUFs  accurately  at  high 
latitude. 

10.  The  use  of  linear  regress  analysis  demonstrated  that  the  source  of  error 
in  HFMUFES  4  predicted  MUFs  (and  to  some  extent  1TSA-1  predicted  MUFs)  was  due 
to  a  bias  in  the  critical  frequency  calculation.  This  bias  occurred  over 
ocean  areas  where  uo  real  vertical  ionosonde  data  were  available  to  generate 
the  numerical  map  of  fQF2.  The  "k  sec  factor  accentuated  the  error  in 
predicted  MUF  as  range  increased. 

11.  For  MINIMUF-3.5  linear  regression,  analysis  showed  that  its  errors  in 
predicted  MUF  at  sunrise  and  for  path  lengths  5000  to  7000  km  were  non-linear 
and  could  be  attributed  to  the  "k  sec  factor  (M  factor)  in  its  calculation 
of  MUF. 

mnora— datxobs 

1.  Use  MINIMUF-3.5  where  the  only  desired  parameters  are  MUF  and  FOT. 

2.  Use  the  universal  time  ionospheric  data  tape  in  NTP  6  Supp.  1  predictions. 

3.  Develop  an  improved  M  factor  equation  for  use  in  MINIMUF-3.5. 

4.  Augment  the  MOF  data  base  to  better  represent  the  Atlantic  Ocean,  northern 
latitudes  and  transequator ial  paths. 

5.  Use  this  augmented  data  base  to  more  accurately  assess  the  errors  in 
predicted  MUF. 

6.  Assess  the  effect  of  the  minimum  take-off  angle  on  accuracy  of  predicted 
MUF. 

7.  Use  multiple- linear  regression  to  remove  the  bias  in  predicted  MUF  due  to 
fQF2  bias. 

8.  In  future  numerical  mapping  of  f qF2  ,  use  topside  sounder  data  to  aid  in 
the  representation  of  over  ocean  areas. 
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INTRODUCTION 


The  effective  operation  of  long  distance  high  frequency  (HF)  systems  has 
increased  in  proportion  to  the  ability  to  predict  variations  in  the  iono¬ 
sphere,  since  such  an  ability  has  permitted  the  selection  of  optimum  frequen¬ 
cies,  antennas  and  other  circuit  parameters.  Most  variations  in  HF  system 
performance  are  directly  related  to  changes  in  the  ionosphere,  which  in  turn 
are  affected  in  a  complex  manner  by  solar  activity,  seasonal  and  diurnal  vari¬ 
ations  as  well  as  latitude  and  longitude. 

Manual  methods  were  developed  for  analyzing  these  effects  on  HF  circuits 
of  short,  intermediate  and  long  distances.1  Because  the  manual  methods  were 
laborious  and  time  consuming,  various  organizations  developed  computer  pro¬ 
grams  to  analyze  HF  circuit  performance.  A  commonly  predicted  parameter  in 
these  programs  is  the  maximum  usable  frequency  (MUF).  The  MUF  is  the  highest 
frequency  that  can  be  propagated  by  ionospheric  refraction  between  given 
points  at  a  given  time. 

There  has  been  very  little  attempt  to  systematically  verify  the  accuracy 
of  HF  prediction  programs.  At  the  Naval  Electronics  Laboratory  Center  (NELC), 
predecessor  to  the  Naval  Ocean  Systems  Center  (NOSC),  three  HF  prediction  pro- 
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grams  were  compared  against  oblique  sounder  data.  MINIMUF-3  was  also  com¬ 
pared  against  the  same  data  base.3  The  results  showed  all  four  programs  to  be 
comparable.  In  the  latter  study  the  data  base  of  oblique  sounder  maximum 
observed  frequency  (MOF)  was  also  enlarged.  This  data  base  consists  of 
measurements  of  median  monthly  MOF  on  25  paths  and  includes  over  4700  hourly 
observed  MOFs.  Geographically,  the  data  base  covers  the  Pacific  Ocean,  Europe 
and  the  continental  United  States  as  well. 

In  this  report  the  accuracy  will  be  discussed  of  predicted  median  MUFs  by 
four  prediction  programs  commonly  used  in  Navy  applications.  The  first  of 
these,  the  ITSA-1  with  local  time  ionospheric  data  tape,  is  used  by  the  Navy 
Electromagnetic  Spectrum  Center  to  produce  its  recommended  frequency  bands 
published  in  Naval  Telecommunications  Publication  6  Supplement  1  (NTP6  Supp. 
1).  This  is  a  Navy  publication  designed  to  provide  the  operating  personnel, 
establishing  ship,  shore  and  aircraft  communication  circuits,  with  a  sample 
reference  for  ionospheric  propagation  predictions  in  the  form  of  recommended 
operating  frequencies.^  The  second  of  these,  the  ITSA-1  with  ionospheric  data 


mapped  in  universal  time,  is  used  to  produce  a  frequency  guide  and  a  frequency 
time  transmission  schedule  for  a  particular  operating  communication  system.^ 
The  third  program,  HFMUFES  4,  is  used  at  NOSC  in  system  design  analysis.  The 
final  program  to  be  compared  is  the  HISIMUF-3.5,  which  has  been  used  ry  NOSC 
in  a  first-generation  forecasting  terminal  called  PROPHET  (a  pseudoacr r-.ym  for 
a  propagation  forecasting  terminal).^  Its  code  is  small  enough  to  be  ised  on 
a  miniconputer. 

The  scope  of  the  report  is  limited  to  the  accuracy  of  the  pred_rted  MUF 
for  point-to-point  paths.  The  accuracy  of  the  predicted  MUF  in  its  applica¬ 
tion  (i.e.,  sector  predictions  rather  than  point-to-point;  seasonal  rather 
than  months,  etc.)  will  be  considered  in  a  subsequent  report.  The  conparison 
is  limited  to  the  existing  data  base  of  oblique  sounder  MOF  data.  In  a  future 
project,  data  for  the  Atlantic  Ocean,  northern  latitude  and  transecuarorial 
paths  will  be  added  to  provide  a  better  balanced  data  base. 
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BISTORT  OF  BXGH  FREQUENCY  PREDICTION 


The  increased  dependence  in  the  past  25  years  upon  high  frequency  telecom¬ 
munication  circuits  has  resulted  in  the  need  for  computer-produced  radio 
predictions.  This  is  especially  true  because  of  the  speed  with  which  modern 
electronic  computers  can  handle  the  large  volumes  of  data  and  can  perform  the 
lengthy  computations.  Many  different  models  of  ionospheric  radio  propagation 
in  all  its  facets  have  been  developed,  ranging  from  extremely  simple  approxi¬ 
mations  to  very  complex  ray-tracing  techniques. 

In  the  United  States,  the  first  automated  HF  path  prediction  conqputer 
program  was  developed  in  1957  for  the  US  Army  Signal  Corps,  Radio  Propagation 
Agency,  now  part  of  the  US  Army  Communications-Electronics  Engineering 
Installation  Agency  (USACEEIA).7  A  later  version  was  published  in  1962  by 
Stanford  Research  Institute  (SRI).8  A  NELIAC  language  version  of  this  program 
was  adapted  to  work  on  NELC's  CDC  1604B  computer  in  1965  and  was  used  until 
1969  when  NELC  changed  to  the  IBM  360/65,  which  didn't  support  the  NELIAC  com¬ 
puter  language.8  This  program  used  ionospheric  data  taken  from  NBS  Technical 
Notes  2  and  2-2;  the  latest  version  of  this  program  also  used  noise  data  from 
CCIR  report  322. 10-12 

The  first  program  to  use  a  numerical  representation  of  the  monthly  fgF2 

and  M-3000  factor  by  tables  of  numerical  coefficients  similar  to  those  in  a 

Fourier  series  was  produced  by  Lucas  and  Haydon  in  196 I.15  Subsequent  to  this 

they  produced  a  program  that  also  calculated  the  field  strength,  the  transmis- 
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sion  loss,  the  available  signal-to-noise  ratio  and  the  circuit  reliability. 

1 2 

At  NEL  this  program  was  modified  to  include  CCIR  Report  322  noise.  The 
Collins  Radio  Conpany  program  produced  in  1963  is  similar  to  that  of  the  CRPL 
and  yields  comparable  data,  but  its  calculation  of  LUF  and  auroral  absorption 
was  different. 15-16  In  1964,  AVCO  Corporation  developed  a  conputer  program  to 
determine  the  possible  ways  by  which  HF  communication  to  and  within  polar 
regions  may  be  maintained  throughout  ionospherically  disturbed  conditions.17 
Their  program  was  intended  only  to  be  used  at  high  sunspot  number. 

The  first  fully  automated  program,  in  which  the  oblique  transmission 
equations  for  parabolic  layers  were  used,  was  developed  in  August  1966  by 
Lucas  and  Haydon  at  Environmental  Sciences  Service  Administration's  (ESSA) 
Institute  for  Telecommunication  Sciences  and  Aeronony  (ITSA).  This  program 
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is  commonly  called  ITSA-1  or  HFMUFS.  It  provides  better  statistical  descrip¬ 
tions  of  the  expected  performance  of  radio  systems  depending  upon  ionospheric 
propagation  of  radio  waves.  The  concepts  of  service  probability  and  reliabil¬ 
ity  are  introduced  in  the  HFMUFS  program.  The  earlier  versions  of  this  pro¬ 
gram  calculated  fgF2  and  the  M(3000)F2  as  a  function  of  local  time  by  the 
method  of  Jones  and  Gallet.^9  Later  versions  of  the  program  calculated  fQF2 
and  M(3000)F2  as  a  function  of  universal  time  using  the  method  of  Jones  et 
al.^®  The  former  method  had  two  major  problems;  (1)  tendency  of  numerical 
maps  to  smooth  out  physical  properties  of  the  ionosphere,  particularly  at  low 
latitudes,  and  (2)  ambiguous  values  at  geographic  poles  and  resulting  distor¬ 
tions  in  immediate  surroundings.  In  the  latter  method,  the  second  problem  was 
overcome  by  means  of  a  universal  time  analysis;  a  significant  improvement  was 
made  in  solving  the  first  problem  by  incorporating  the  effect  of  the  magnetic 
field  variations.  The  version  of  the  ITSA-1  program,  obtained  in  May  1968  at 
NELC  and  now  in  use  at  NOSC,  uses  the  new  Jones  method. 

DNC-14,  the  Navy's  recommended  frequency  bands  and  frequency  guide  for 

operating  personnel  establishing  ship,  shore  and  aircraft  cczmur.ication 

circuits,  was  adapted  on  the  IBM  7090  from  the  ESSA-ITSA-1  program  ir.  August 

1966.  The  data  tape  containing  the  numerical  coefficients  of  f,jF2  and 
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M(3000)F2  uses  the  earlier  local  time  method  of  Jones  and  Gallet.  This  data 
tape,  dated  August  1966,  was  never  replaced  by  a  tape  containing  the  new  coef¬ 
ficients  because  CNO  (OP-094F)  was  considering  the  possibility  of  adapting  the 
DNC-14  output  into  the  newer  ITS- 78  program  and  because  DNC-14  itself  was 
being  converted  to  the  Univac  1108  computer.  When  the  DNC-14  was  converted  to 

the  Univac  1108  on  November  1972,  the  old  August  1966  tape  was  retained.  The 

4 

results  using  this  program  are  promulgated  in  NTP  6  Supp.  1 . 

Bell  Aerosystems  Conqgany  in  1967  developed  a  program  for  Rome  Air  Devel¬ 
opment  Center  ( RADC)  to  provide  loss  calculations  for  RADC's  interference 
prediction  coiqputer  program.  Raytracing  equations  were  used  to  obtain  an 
output  directly  applicable  to  interference  analysis.  Techniques  employed  in 
this  model,  unlike  most  models,  identified  possible  stable  and  unstable  modes, 
including  mixed  modes.  The  ionospheric  data  used  were  the  same  as  in  the 
latest  ESSA-ITSA  version.  Modes,  radiation  angles  and  losses  wer^  given  for 
the  different  probability  levels  of  ionospheric  support. 
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In  1969  Barghausen  et  al,  at  the  Institute  for  Telecommunication  Sciences 
(ITS),  developed  a  program,  commonly  called  ITS-78  (Red  Deck)  or  HFMUFES, 

which  employed  more  extensive  techniques;  though  similar  to  that  in  the  ITSA-1 
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program,  it  incorporated  significant  changes.  The  major  changes  are: 

(a)  All  numerical  coefficients  representing  the  ionospheric  character¬ 
istics  are  calculated  as  functions  of  universal  time 

(b)  E-layer  propagation  characteristics  are  calculated  from  numerical 
coefficients  representing  the  E-layer  critical  frequencies 

(c)  Numerical  coefficients  representing  the  minimum  virtual  height  of  the 
F-region  have  been  included  for  calculating  the  semi-thickness  of  the  F-layer 

(d)  Revised  values  of  man-made  noise  and  its  frequency  dependence  have 
been  included 

(e)  A  method  for  combining  two  or  more  noise  sources  of  nearly  equal 
amplitudes  has  been  added 

(f)  A  new  formula  is  used  for  estimating  absorption,  based  upon  extensive 
measurements,  and  including  a  winter  anomaly  effect 

(g)  System  performance  predictions  can  be  made  for  sporadic-E  propagation 

(h)  The  chi-square  probability  distribution  is  used  to  evaluate  all 
distributions 

(i)  Revised  excess  system  losses  have  been  included 

In  December  1970  ITS  issued  a  revised  version  of  the  ITS-78,  commonly 
called  either  the  Blue  Deck  or  HFMUFES  2.  This  program  followed  closely  the 
models  and  methods  described  in  ITS-78,  except  as  outlined  briefly  below: 

(a)  Numerical  coefficients  representing  the  F2  layer  critical  frequency 
as  a  function  of  latitude,  longitude  and  time  were  revised  to  include  the 
solar  cycle  and  seasonal  variations  of  fjF22^ 

(b)  Numerical  coefficients  representing  atmospheric  noise  as  a  function 
of  universal  time  were  included^4 

(c)  The  numerical  maps  of  the  representing  minimum  virtual  height  were 
revised 

(d)  Some  revision  was  made  to  the  --orld  n»ps  representing  fyE25 
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(e)  Provision  was  made  to  use  up  to  three  different  transmitting  and 
receiving  antennas  over  the  HF  band. 

In  December  1974  ITS  issued  a  revised  version  of  its  ITS-78  line  of  com¬ 
puter  programs,  commonly  called  either  the  yellow  deck  or  HFMUFES  3.  Mainly, 
this  version  contained  corrections  to  the  code  to  remove  known  errors  in  the 
programming.  Further,  this  version  was  the  first  of  the  HFMUFES  series  to 
allow  input  of  antenna  pattern  data  via  magnetic  tape. 

Finally,  in  September  of  1976  ITS  issued  HFMUFES  4,  the  current  version 

26 

of  the  HFMUFES  series  of  programs.  The  main  difference  between  this  program 
and  earlier  versions  is  that  sporadic-E  is  no  longer  considered  in  the  calcu¬ 
lation  of  the  MUF  and  reliability.  In  the  earlier  versions,  sporadic-E  was 
allowed  as  an  option. 

In  1978  NOSC  developed  a  simplified  HF  MUF  prediction  algorithm  called 
MINIMUF-3.3  It  was  designed  to  complement  existing  large-scale  HF  propagation 
codes  when  computation  resources  were  limited  and  large-scale  codes  were  not 
feasible  to  execute.  It  was  based  on  the  idea  that  fQF2  can  be  modeled  to  a 
first  approximation  as  the  logged  response  to  a  driving  function  proportional 
to  (cos  x)r‘  where  x  is  the  instantaneous  solar  zenith  angle  and  when  the  day¬ 
time  lag  is  quite  seasonally  dependent.  It  was  shown  to  be  sufficiently  accu¬ 
rate  to  provide  a  MUF  prediction  suitable  for  use  on  small  mobile  propagation 
forecast  (PROPHET)  terminals.  The  most  current  model,  called  MINIMUF-3. 5, 
allows  MINIMUF-3  to  be  used  out  to  the  antipodal  point. ^  MINIMUF-3  was  con¬ 
strained  to  be  used  in  the  800-8000  km  range.  MINIMUF-3. 5  has  been  compared 
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against  short  range  (  192  km  and  433  km)  oblique  sounder  data.  Predictions 
for  the  433  km  path  are  reasonably  close  to  observed  mean  MOFs.  There  is  some 
over-prediction  during  the  daytime,  but  very  close  correlation  at  night.  How¬ 
ever,  the  data  for  the  192  km  path  displayed  a  drastic  MOF  depression,  caused 
by  E-region  cutoff.  In  this  case,  MINIMUF-3. 5  was  inadequate. 


COM* ARISON  PROCEDURE 


To  develop  a  sufficient  knowledge  about  the  capabilities  of  the  four  pro¬ 
grams  being  conpared,  they  were  conpared  against  swept  frequency  HF  oblique 
sounder  data.  A  MOF  data  base  previously  assembled  for  the  MINIMUF  develop¬ 
ment  task  was  used.  The  results  will  indicate  how  the  models  perform  when 
correlated  with  "real-world"  propagation  and  will  provide  a  relative  indica¬ 
tion  of  the  differences  between  a  first-order  approximation  and  more  sophisti¬ 
cated  prediction  codes. 

OBLIQUE  SOUNDER  DATA  BASK  PREPARATION 

The  oblique  sounder  data  base  that  was  assembled  was  derived  from  a  vari¬ 
ety  of  sources  and  spans  the  period  between  1960  and  1976.  This  represents 
over  one  complete  solar  sunspot  cycle  of  propagation  data.  Attempts  were  made 
to  make  the  data  base  as  diverse  as  possible  including  a  variety  of  different 
path  lengths,  orientations  and  geographical  locations.  While  measurements 
from  several  different  types  of  oblique  sounder  systems  were  included,  the 
majority  of  data  came  from  the  Navy  Tactical  Sounder  (NTSS). 

Navy  Tactical  Sounder  System  (NTSS) 

The  Navy's  oblique  ionospheric  sounder  system  consists  of  several  shore- 
based  sounder  transmitters  and  a  number  of  sounder  receivers.  AN/FPT-11  (XN- 
1)  sounder  transmitters  were  installed  at  selected  Naval  communication  sta¬ 
tions.  The  system  receiver  and  an  AN/UPR-2  receiver  were  installed  at 
selected  Naval  communications  stations,  research  installations  and  aboard 
ships. 

Once  each  minute  the  FPT-11  transmitter  sequentially  transmitted  a 
double,  biphase,  Barker-coded  pulse  on  each  of  80  discrete  frequencies  between 
2  and  32  MHz;  the  total  scan  consisted  of  160  pulses  lasting  16  seconds.  The 
frequency  range  is  divided  into  four  octave  bands,  with  20  channels  linearly 
spaced  in  each  band.  The  80  frequencies  were  spaced  in  100  kHz  increments  in 
the  2  to  4  MHz  range  ( Band  A),  230  kHz  increments  from  4  to  8  MHz  (Band  B), 
400  kHz  increments  from  8  to  16  MHz  (Band  C)  and  800  kHz  increments  from  16  to 
32  MHz  (Band  D). 


7 


F? 

i 


The  UPR-2  receiver  sequentially  processed  the  pulse-train  input  by  start¬ 
ing  the  gated  receiver  scan  at  the  same  time  as  the  transmission.  This  was 
accoag>lished  by  synchronizing  to  a  common  timing  source  (i.e.,  WWV)  and  main¬ 
taining  an  accurate  time  base  generator  in  the  receiver.  Since  each  sounder 
signal  is  con^osed  of  a  series  of  13  Barker-coded  subpulses,  signal  processing 
is  required  in  the  receiver.  The  process  gain  over  noise  is  11  dB.  A  perma¬ 
nent  record  of  the  daily  variations  of  the  scanned  spectrum  between  2  and  32 
MHz  is  produced  on  strip  charts.  To  supplement  this  capability,  NOSC  devel¬ 
oped  a  method  of  digitizing  the  video  output  signal  and  recording  it  on  mag¬ 
netic  tape.  The  HP  digital  recorder  ( HFDR)  developed  for  this  purpose 
operates  concurrently  with  the  AN/UPR-2  receiver  and  in  no  way  affects  normal 
operation.  Hence,  with  the  HFDF  equipped  sounding  receiving,  all  amplitude, 
time  delay  and  frequency  information  are  recorded  once  every  minute,  24  hours 
a  day. 

Other  Sounder  Systems 

Data  collected  prior  to  1968  were  measured  on  a  variety  of  sounder  sys¬ 
tems.  One  system,  used  primarily  by  SRI,  used  the  Model  900  series  of 
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sounders  made  by  Granger  Associates.  These  scanned  the  range  of  frequencies 
from  approximately  4  to  64  MHz  in  4  one-octave  bands  of  40  linearly  spaced 
channels  each.  The  transmitted  output  is  in  pulses  of  0.1  msec  (short  pulse) 
or  1.0  msec  (long  pulse)  at  30-kw  peak  amplitude,  repeated  2  or  4  times  each 
channel.  The  long  pulse  is  more  appropriate  for  communication  system  sounding 
and  also  presents  a  higher  average  power  which  is  often  needed  on  long  paths. 
The  short  pulse  is  used  for  mode  resolution  and  is  normally  made  as  narrow  as 
possible  within  the  limitations  set  by  the  length  of  the  sounded  path.  The 
entire  scan  was  conpleted  in  29  seconds  and  was  repeated  every  20  minutes. 
Another  sounder  system,  a  modified  C-3  ionosonde  transmitted  0.1  msec  pulses; 
the  transmitting  frequency  was  swept  linearly  between  2  and  25  MHz.3®  In  some 
instances  data  were  acquired  by  means  of  a  Granger  transmitter  and  UPR-2 
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Data  Categorization 

i 

The  source  of  the  oblique  sounder  data  is  important  because  it  influences 

i 

the  statistical  significance  of  a  given  path-month.  The  ojverall  sounder  data 
were  categorized  into  five  sources: 

a.  NTSS-HFDF  j 

b.  NTSS-strip  chart 

c.  Non-NTSS 

I 

d.  Granger  900  series 

e.  Modified  C-3 

A  path-month  MOF  ourve  from  the  NTSS-HFDF  system  is  generally  the  product 
of  approximately  40,000  digitally  processed  measurements J ( up  to  1861  an  hour 
over  the  month).  Th*  resolution  of  the  NTSS-strip  chart  System  limits  this  to 
about  2880  hand-seal":  1'ta  points  per  path-month  (120  per  hour  of  the  month). 
The  Granger  seri-»j  data  consisted  of  three  scans  per  hour  or  90  per  hour  of  a 
month  and  2160  per  path-month.  The  modified  C-3  data  consisted  of  one  7.5 
minute  sweep  e  'ery  hour.  This  was  equivalent  to  720  per  path-month  (30  per 
hour  per  month).  The  non-NTSS  system  consists  of  180  points  per  hour  or  4320 
data  points  per  month. 

The  data  can  also  be  categorized  according  to  the  frequency  range  of  the 
sounder  transmitter.  In  the  first  three  categories,  the  sounder  scanned  the 
range  from  2  to  32  MHz.  The  Granger  900  series  scanned  the  range  from  4  to  64 
MHz,  and  the  modified  C-3  scanned  the  range  from  2  to  25  MKz . 

DESCRIPTION  OP  OVERALL  SOUNDER  DATA  BASS 

The  final  oblique  sounder  data  set  consisted  of  198  path-months  of  median 
hourly  MOF  values  derived  from  25  different  HF  transmission  paths.  The  long¬ 
est  path  was  7808  km  and  the  shortest  path  was  192  km.  The  set  contains  a 
cross-section  of  transmission  paths  including  mid-latitude,  transauroral, 
transequatorial,  all  seasons  and  all  solar  sunspot  numbers  (SSN).  Table  1 
summarizes  the  basis  against  which  the  four  programs  were  cotqpared.  The  loca¬ 
tions  of  the  paths  are  shown  in  Figure  1  except  for  the  two  shortest  paths 
(the  scale  is  tcc  small  to  illustrate  them). 
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DATA  SCREENING 


In  the  comparison  of  the  program,  it  is  highly  desirable  to  subdivide  the 
data  base  into  subsets  according  to  variables  influencing  the  predicted  and 
observed  results  (e.g. ,  path  length,  season,  nvonth,  geomagnetic  latitude,  sun¬ 
spot  number,  local  time  at  path  midpoint,  etc.  ) .  To  accomplish  this  a  com- 
0-  puter  program  called  DASCR3  (acronym  for  data  screening  3)  was  used.  Each  of 

the  prediction  programs  was  run  for  each  of  the  paths  in  Table  1.  The  results 
along  with  auxiliary  information  about  the  propagation  situation  (e.g.,  path 
length,  local  time  of  day,  sunspot  number,  etc.)  were  stored  in  a  data  file  to 
be  used  later  by  DASCR3. 

DASCR3 

DASCR3  is  a  program  designed  to  perform  data  screening  and  statistical 
comparison  on  two  large  matrices  of  observations.  For  each  set  of  matrices, 
up  to  10  sets  of  information  are  read  in  on  propositions  to  be  satisfied  and 
limits  on  a  selected  variable.  A  portion  of  each  matrix  is  read  m  and  tested 
for  each  set  of  propositions  in  turn.  For  each  subset  satisfying  a  giver,  set 
of  conditions,  the  variable  to  be  analyzed  is  stored  temporarily  on  disc.  The 
next  portion  of  each  matrix  is  then  read  in  and  screened  and  the  good  observa¬ 
tions  are  added  to  those  already  on  disc.  When  the  entire  matrix  has  beer, 
screened  the  screened  data  are  then  read  into  core  and  the  difference  (or 
residual)  between  the  two  matrices  is  taken.  These  arrays  are  then  sorted  to 
ensure  maximum  computer  efficiency  for  the  statistical  evaluation.  Finally,  a 
statistical  evaluation  is  then  performed  of  the  screened  data  and  their 
residuals. 

An  example  of  the  output  from  DASCR3  is  given  in  Figure  2.  In  this 
sample  the  ITSA-1  program,  using  the  universal  time  set  of  numerical  coeffi¬ 
cients,  is  compared  to  the  observed  data.  The  proposition  to  be  satisfied  is 
that  the  data  to  be  evaluated  be  for  month  equal  1  (January).  The  variables 
being  compared  are  the  observed  MOF  and  predicted  MUF.  In  the  printout  the 
observed  data  are  represented  by  column  A  and  the  predicted  values  are  repre¬ 
sented  by  column  B.  The  residual  (the  observed  data  minus  the  predicted 
value)  is  given  by  column  D.  The  relative  residual  is  given  by  column  D/A, 
and  the  absolute  relative  residual  by  column  ABS(D)/A.  The  left  hand  side  of 
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H'iyure  2.  Example  output  from  DASCR3  , 


the  page  shows  the  statistics  calculated  for  each  of  these  columns.  In  addi¬ 
tion  the  correlation  coefficient  between  the  observed  and  predicted  data  are 
given.  Included  also  are  the  slope,  intercept  and  rtvean  square  error  of  linear 
regression.  In  this  example  288  data  points  were  selected  by  DASCR3  from  4668 
data  points.  Note  that  the  average  absolute  relative  residual  for  this  case 
is  25.9  percent. 

Screening  Data  Base 

Each  computer  program  was  run  to  produce  a  data  base  corresponding  to  the 
observed  data  base.  Auxilliary  information  outputted  to  be  screened  in¬ 
cluded:  universal  time  of  propagation,  month,  year,  sunspot  number,  path 
length  in  kilometers,  geographic  latitude  and  longitude  of  the  path  midpoint, 
the  local  time  at.  the  path  midpoint,  the  path  orientation  with  respect  to 
north,  the  geomagnetic  latitude  at  each  of  the  control  points,  the  predicted 
MUF,  E-layer  MUF,  F-layer  MUF,  FOT,  HPF  and  path  identification  number  and 
sounder  type. 

Before  the  actual  data  screening  was  begun,  data  points  in  both  observed 
and  predicted  bases  corresponding  to  observed  values  at  the  extremes  of  the 
particular  measuring  sounder  were  removed  from  the  data  base.  The  final  num¬ 
ber  of  hourly  values  in  the  data  base  was  4668  points. 


ANALYSIS  OF  RESIDUALS  BETWEEN  PREDICTIONS  AND  OBSERVED  DATA 

An  indication  of  the  accuracy  of  the  numerical  predictions  o'  MUF  c*. 
obtained  from  a  study  of  the  residuals  between  observed  data  and  predicted 
values.  The  terms  residual,  relative  residual  and  absolute  relative  residual 
are  used  with  the  following  standard  meaning: 


residual  =  (observed  datum)  -  (predicted  value) 


relative  residual  = 


residual 


observed  datum 


absolute  relative  residual  =  absolute  residual 

observed  datum 


(1) 

(2) 

(3) 


Certain  statistical  measures  of  these  terms  have  proved  useful  in  past  iono- 
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spheric  studies  in  comparing  predicted  and  observed  data. 


These  include: 


(1)  The  average  residual  (av.  res.) 

(2)  Root  mean  square  residual  (rms  res.) 

(3)  The  mean  absolute  error  of  the  residual  (mae  res.) 

(4)  The  average  relative  residual  (av.  rel.  res.) 

(5)  The  root  mean  square  relative  residual  (rms  rel.  res.) 

(6)  The  mean  absolute  error  of  the  relative  residual  (mae  rel.  res.  ) 

(7)  The  average  absolute  relative  residual  (ave.  abs.  rel.  res.) 

(8)  Correlation  coefficient  between  observed  and  predicted  values 

(9)  The  standard  error  of  the  estimate  of  linear  regression. 

Values  of  each  of  these  parameters  are  produced  by  DASCR3  as  can  be  seen  by 
examining  Figure  2. 

The  average  residual  and  the  average  relative  residual  locate  the  center 
of  the  distributions  of  error  and  are  sometimes  referred  to  as  the  bias  in  the 
estimate.  Figures  3  and  4  illustrate  the  average  residual  and  average  rela¬ 
tive  residual,  respectively,  as  a  function  of  month  for  the  four  programs 
being  compared.  In  this  example,  KINIMUF-3.5  is  shown  to  have  the  smallest 
bias;  whereas,  HFMUFES  4  tends  to  always  predict  high  by  as  much  as  3.5  MHz  or 
17.5  percent. 

The  mean  absolute  errors  of  th  residual  and  relative  residual  are  a  mea¬ 
sure  range  of  the  error  and  are  the  first  moments  about  the  average  residual 
and  average  relative  residual,  respectively.  They  provide  information  about 
the  range  of  variation.  Figures  5  and  6  are  examples  of  these  two  parameters, 
respectively,  for  MINIMUF-3.5.  They  are  displayed  as  bars  about  the  average 
residual  (bias)  as  a  function  of  month.  The  mae  of  the  residual  is  rather 
uniform  as  a  function  of  month.  However,  Figure  6  shows  that  the  range  of 
variation  in  the  error  during  the  equinox  months  March  and  September  to  be 
greater  than  the  other  months. 

The  average  absolute  relative  residual  is  a  measure  of  the  average 
magnitude  of  the  error.  Figure  7  shows  a  plot  of  the  average  absolute  rela¬ 
tive  residual  as  a  function  of  month  for  the  four  programs  being  compared. 


Figure  3.  Average  residual  (bias)  as  a  function  of  month. 
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Figure  4.  Average  relative  residual  (relative  bias)  as  a  function  of  month 


Figure  5.  Average  residual  (Mas)  for  MINIMUF-3.5  with  the  mean  absolute 
error  about  the  average  residual. 
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Figure  7.  Magnitude  of  the  error  (average  absolute  relative  residual)  as 
function  of  month  . 


The  root  mean  square  residual  and  relative  residuals  are  treasures  of  the 
dispersion  in  the  error.  In  fact,  the  rms  residual  and  rms  relative  residual 
are  the  standard  deviations  of  the  error  about  the  origin  (zero  bias)  and  are 
related  to  the  standard  deviation  about  the  mean  according  to 


where  Vj  the  mean  square  error  (the  square  of  the  rms  error)  and  is  the 
bias.  When  the  bias  is  small  or  nearly  zero,  then  the  standard  deviation  and 
the  rms  error  are  nearly  the  same.  Otherwise,  the  rms  error  is  larger  than 
the  standard  deviation.  Figures  8  and  9  are  examples  of  the  rms  residual  and 
rms  relative  residual,  respectively,  for  the  four  programs  being  compared  as  a 
function  of  month.  MINIMUF-3.5  has  the  lowest  rms  error  reaching  its  highest 
value  of  4  MHz  plus  (12  percent)  during  October;  whereas,  HFMUFES  4  has  its 
lowest  values  during  the  summer  months  and  has  the  highest  rms  error  during 
the  winter  months. 

A  measure  of  the  degree  of  association  or  the  closeness  of  fit  between 
variables  is  given  by  the  correlation  coefficient.  It  indicates  the  strength 
of  the  tendency  for  high  (or  low)  values  of  one  variable  to  be  associated  with 
high  (or  low)  values  of  the  other  variable.  Figure  10  is  an  example  of  the 
correlation  coefficients  for  the  four  programs  being  compared  as  a  function  of 
month.  In  this  example  HFMUFES  4  generally  has  the  highest  correlation  coef¬ 
ficient  with  MINIMUF-3.5  also  showing  consistently  high  values. 

A  description  of  the  nature  of  the  relationship  between  variables  is 

3  8 

called  regression  analysis.  Regression  analysis  is  concerned  with  the  prob¬ 
lem  of  describing  or  estimating  the  value  of  one  variable,  called  the  depend¬ 
ent  variable,  on  the  basis  of  one  or  more  other  variables,  called  independent 
variables.  In  other  cases  regression  may  be  used  merely  to  describe  the  rela¬ 
tionship  between  known  values  of  two  or  more  variables. 

Regression  analysis  that  involves  the  determination  of  a  linear  relation¬ 
ship  between  two  variables  is  referred  to  as  simple  linear  regression.  Here, 
the  variable  y  is  given  as  y  =  a  +  bx  where  x  is  the  independent  variable  and 
y  is  the  dependent  variable.  The  coefficients  a  and  b  are  determined  in  the 
regression  analysis.  A  measure  of  the  success  of  linear  regression  analysis 
is  the  standard  error  of  the  estimate  given  by 
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Figure  H.  Rms  error  in  MHz  ns  a  function  of  month. 


Fiqure  9.  Rms  relative  error  in  percent  as  a  function  of  month. 


(5) 


S 

y.x 


where  is  the  standard  deviation  in  the  observed  datum  and  Y  is  the  correla¬ 
tion  coefficient  between  the  observed  data  and  predicted  values.  If  the  rela¬ 
tionship  is  truly  linear,  then  the  bias  of  the  estimate  should  be  removed  (or 
made  nearly  zero).  An  estimate  of  the  standard  error  of  mean  is  given  by 

S-  =  (6) 

y*X  ^ 

A  measure  of  the  error  in  the  regression  coefficient  is  given  by 

S-  1/2 

sb  =  /[-X-_j 

X 

where  the  standard  deviation  in  the  predicted  values.  Figures  11  and  12 
show  the  standard  error  of  the  estimate  of  linear  regression  and  of  standard 
error  of  mean  in  linear  regression,  respectively,  as  a  function  of  month. 
When  Figure  11  is  conpared  to  Figure  8,  the  largest  change  occurs  for  HFMUFES 
4.  Very  little  change  is  shown  for  ITSA-1  with  local  time  tape.  MINIMUF-3.5 
shows  some  changes  for  some  months  but  not  all.  Figure  12  shows  that  linear 
regression  has  removed  much  of  the  bias  in  the  predicted  MUFs. 


COMPARISON  RESULTS 

This  section  will  present  the  results  MUF  prediction  comparisons.  The 
obiective  is  to  provide  the  reader  with  a  clear  understanding  of  the  capabili¬ 
ties  and  limitations  of  each  prediction  model. 


ALL  CASES 

The  oblique  sounder  data  base  consisted  of  198  ptt-h-months  of  observed 
MOFs  taken  over  25  transmission  paths.  The  shortest  path^  was  196  km  and  the 
longest  path  was  7808  km.  For  each  program  being  compared,  Tables  2-5  list 
the  paths  and  the  bias,  the  rms  error,  the  average  magnitude  of  the  error,  and 
the  correlation  coefficient  between  the  MOFs  and  MUFs  for  each  path.  Table  2 
shows  MINIMUF  3.5  to  have  zero  percent  bias  tor  three  paths  and  to  predict 
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Figure  11.  Standard  rms  error  of  estimate  of  linear  regression  as  a  functi< 
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No. 

Transmission  Path 

Bias , 

MHz 

rjis, 

>'  H  2 

error , 

% 

Magni¬ 
tude,  ‘ 

1 

Guam  to  Yokohama,  Japan 

0  .03 

0  .0 

5.15 

8.9 

27.5 

2 

FT  Monmouth,  NJ  to  Palo  Alto,  CA 

1.73 

8.4 

4.12 

11.2 

19.2 

3 

Guam  to  Honolulu,  Hawaii 

0  .34 

1  .0 

3  .00 

4.2 

14.5 

4 

Guam  to  Kodiak,  Alaska 

-2.14 

-10.5 

3.87 

14.5 

22.2 

5 

Honolulu,  Hawaii  to  Kodiak,  Alaska 

0  .0  8 

-0  .0 

2  .40 

5.8 

12.2 

6 

Honolulu,  Hawaii  to  Washington,  DC 

2.24 

13.7 

3.61 

15.7 

14.7 

7 

Davis,  CA  to  Honolulu,  Hawaii 

-1.14 

-5.3 

3  .29 

9  .5 

19  .0 

8 

Palo  Alto,  CA  to  Fairbanks,  Alaska 

0.83 

3.2 

2.17 

9.7 

9.3 

9 

Boulder,  CO  to  Pt.  Barrow,  Alaska 

-0.01 

0  .0 

4  .21 

8.5 

20  .2 

10 

Honolulu,  Hawaii  to  Yokohama,  Japan 

-2.31 

-11.8 

4.32 

12.4 

23.7 

1  1 

Tarlac,  Philippines  to  Yokohama,  Japan 

-0  .33 

-0  .0 

3  .55 

13.1 

13.3 

12 

Tarlac,  Philippines  to  H.E.  Holt, 
Australia 

1 .48 

6.8 

4.31 

10.3 

14.4 

13 

Guam  to  H.  E.  Holt,  Australia 

-1.55 

-5  .2 

4  .67 

10  .2 

19  .0 

14 

Davis,  CA  to  Kodiak,  Alaska 

-2.07 

-13.1 

5.29 

16.9 

33.2 

15 

Honolulu,  Hawaii  to  Corona,  CA 

1  .25 

1  .9 

3  .33 

1 1  .2 

15.1 

1 6 

Andoya,  Norway  to  Thessoloniki,  Greece 

.94 

2.36 

8.7 

15.4 

17 

Davis,  CA  to  La  Posta,  CA 

1  .40 

15  .0 

2  .46 

19  .4 

17.8 

18 

Toulouse,  France  to  Neimakri,  Greece 

-2.99 

-29 .4 

4.05 

35.2 

42.1 

19 

Honolulu,  Hawaii  to  La  Posta,  CA 

-0.79 

-4.9 

2  .70 

10  .6 

18.8 

20 

Coco  Solo,  Canal  Zone  to  Stockbridge, 

NY 

1 

o 

tn 

O 

-2.3 

2.7a 

5.8 

12.1 

21 

Andoya,  Norway  to  New  Delhi,  India 

2.17 

14.8 

3.00 

15.7 

14.9 

22 

Palo  Alto,  CA  to  Thule,  Greenland 

2.77 

18.3 

3.33 

19.5 

16.9 

23 

Toulouse,  France  to  Keflavik,  Iceland 

1  .42 

1  1  .0 

2  .46 

13.3 

1  1  .8 

24 

FT  Monmouth,  NJ  to  Aberdeen,  NY 

-1.46 

-18.3 

1.89 

21.7 

23 .6 

25 

FT  Monmouth,  NJ  to  Camp  Drum,  NY 

-1.17 

-14.8 

1  .6 1 

15.3 

20  .9 

Table  2.  MINIMUF  3.5  c 

omparison 

by  sour. 

der  pa 

th. 

Corre¬ 

lation 

Coeffi¬ 

cient 


0  .665 
0.818 
0  .884 
0 .872 
0  .928 
0  .881 
0.861 
0 .747 
0  .385 
0.862 
0  .877 

0.767 
0  .79  7 
0.713 
0  .947 
0  .933 
0  .734 
0.820 
0  .827 

0  .933 
0 .391 
0.913 
0  .896 
0 .806 
0.875 
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29.4  percent  too  high  for  the  Toulouse,  France,  to  Nermakri,  Greece  path. 
This  same  path  had  the  largest  rms  relative  residual  (35.2  percent)  and  the 
largest  average  magnitude  of  the  error  (42.1  percent).  The  paths  with  highest 
correlation  coefficients  (Honolulu,  Hawaii,  to  Kodiak,  Alaska;  Honolulu, 
Hawaii,  to  Corona,  California;  Andoya,  Norway,  to  Thessoloniki,  Greece;  and 
Coco  Solo,  Canal  Zone,  to  Stockbridge,  New  York)  all  have  low  bias,  rms  error, 
and  magnitude  of  error.  Table  3  shows  HFMUFES  4  to  generally  have  larger  bias 
than  MINIMUF-3.5,  but  has  more  paths  with  correlation  coefficient  greater  than 
0.9  than  does  MINIMUF-3.5.  Tables  4  and  5,  for  the  two  versions  of  ITSA-1, 
show  them  to  have  slightly  higher  bias  than  MINIMUF-3.5  but  also  to  be 
slightly  higher  correlated  with  the  observed  data  than  MINIMUF  3.5. 

Table  6  shows  the  overall  comparison  results.  When  compared,  overall 
MINIMUF-3.5  and  the  two  ITSA-1  versions  can  be  separated  by  only  a  few  tenths 
of  a  percent  in  bias  and  rms  error  and  by  just  one  percent  in  magnitude  of  the 
error.  However,  HFMUFES  4  clearly  has  the  largest  bias  (7.2  percent  high), 
rms  error  (8.3  percent),  and  magnitude  of  error  (26.0  percent).  Table  6  also 
shows  that  HFMUFES  4  has  the  highest  value  of  correlation  coefficient  and  that 
the  largest  reduction  in  rms  using  linear  regression  is  achieved  by  HFMUFES  4 
(rms  error  of  4.24  MHz  reduced  to  a  standard  error  of  estimate  of  3.50  MHz). 
This  indicates  that  the  relationship  between  predicted  results  by  HFMUFES  4 
and  the  observed  MOFs  is  more  nearly  linear  than  the  other  programs  and  that 
its  errors  may  be  due  in  part  to  some  simple  reason. 


DATA  TYPE 

A  critical  part  of  any  investigation  involving  the  use  of  observed  meas¬ 
urements  is  t.he  quality  and  time  resolution  of  the  measurements.  This  is 
particularly  important  when  multiple  samples  are  merged  into  mean  values,  as 
was  the  case  with  the  oblique  sounder  data.  As  discussed  in  the  section  on 
data  preparation,  there  were  five  types  of  sounder  data  used:  (1)  NTSS-HFDR, 
(2)  NTSS-strip  chart,  (3)  non-NTSS,  (4)  Granger  900  series  and  (5)  modified  C- 
3.  The  number  of  data  points  per  hour  per  month  determining  the  hourly 
medians  were:  (1)  160,  (2)  four,  (3)  six,  (4)  three,  and  (5)  one  for  the  five 
data  categories,  respectively.  Table  7  gives  sample  percentage  for  each  data 
category. 
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Corre¬ 

lation 


No. 

Transmission  Path 

Bias, 

MHz 

% 

rms, 

MHz 

error, 

% 

Magni¬ 
tude  ,  % 

Coeffi¬ 

cient 

1 

Guam  to  Yokohama,  Japan 

-3.39 

-18.5 

6.75 

19  .4 

41 .6 

0  .69  7 

2 

FT  Monmouth,  NJ  to  Palo  Alto,  CA 

-0.22 

-4.1 

2.82 

11.0 

16.7 

0  .90  1 

3 

Guam  to  Honolulu,  Hawaii 

-0  .99 

-3.1 

3.31 

7.6 

18.5 

0.924 

4 

Guam  to  Kodiak,  Alaska 

-3.14 

-18.8 

4.10 

21.1 

24.9 

0  .886 

5 

Honolulu,  Hawaii  to  Kodiak,  Alaska 

-1  .86 

-12.7 

4.02 

17.5 

23.1 

0  .834 

6 

Honolulu,  Hawaii  to  Washington,  DC 

4.39 

23.3 

5.36 

23.9 

23.1 

0  .812 

7 

Davis,  CA  to  Honolulu,  Hawaii 

-0  .08 

0  .5 

2.81 

4.7 

14.0 

0  .885 

8 

Palo  Alto,  CA  to  Fairbanks,  Alaska 

3.12 

-15  .9 

3.65 

16.6 

15.5 

0.777 

9 

Boulder,  CO  to  Pt.  Barrow,  Alaska 

0  .28 

-0  .5 

1  .99 

6.7 

9.5 

0 .796 

10 

Honolulu,  Hawaii  to  Yokohama,  Japan 

-2.00 

-10  .2 

3.81 

11.7 

20.8 

0 .898 

11 

Tarlac,  Philippines  to  Yokohama,  Japan 

-1  .05 

-3.8 

2  .46 

6.1 

8.7 

0  .844 

12 

Tarlac,  Philippines  to  H.E.  Holt, 
Australia 

-3.34 

-12.6 

4.39 

13.3 

17.6 

0 .899 

13 

Guam  to  H.  E.  Holt,  Australia 

0  .90 

3.7 

2.45 

4.4 

8.3 

0  .909 

14 

Davis,  CA  to  Kodiak,  Alaska 

-1.25 

-8.4 

5.66 

11.6 

30 .5 

0.656 

15 

Honolulu,  Hawaii  to  Corona,  CA 

5.67 

32  .7 

2.44 

14.2 

52  .2 

0  .929 

16 

Andoya,  Norway  to  Thessoloniki,  Greece 

-1.52 

-13.1 

2.00 

14.8 

16..' 

0  .975 

17 

Davis,  CA  to  La  Posta,  CA 

1  .66 

17.0 

2  .62 

20  .4 

19  .0 

0  .739 

18 

Toulouse,  France  to  Neimakri,  Greece 

-4.79 

-49  .3 

6.04 

50.7 

65.2 

0  .639 

19 

Honolulu,  Hawaii  to  La  Posta,  CA 

-3.75 

-26.0 

4.96 

26.7 

37.5 

0  .816 

20 

Coco  Solo,  Canal  Zone  to  Stockbridge, 

NY 

-1.15 

-4.4 

2.71 

6 . 5 

12.9 

0  .961 

21 

Andoya,  Norway  to  New  Delhi,  India 

1  .96 

12  .2 

2.68 

12.7 

13.2 

0  .912 

22 

Palo  Alto,  CA  to  Thule,  Greenland 

2.46 

1 1 .9 

3.67 

20 .5 

21.7 

0 . 864 

23 

Toulouse,  France  to  Ke.flavik,  Iceland 

0  .69 

5.2 

2.60 

7.0 

15.6 

0  .836 

24 

FT  Monmouth,  NJ  to  Aberdeen,  NY 

0.82 

11.1 

0  .95 

11.7 

10.3 

0  .861 

25 

FT  Monmouth,  NJ  to  Camp  Drum,  NY 

0  .30 

3.0 

0  .71 

5.0 

7.9 

0  .945 

Table  3.  HFMUFES  4  conparison  by  sounder  path 


Corre¬ 

lation 


No. 

Transmission  Path 

Bias, 

MHz 

% 

rms, 

MHz 

error, 

% 

Magni¬ 
tude,  % 

Coeffi¬ 

cient 

1 

Guam  to  Yokohama,  Japan 

-3.16 

-17.8 

6.67 

18.9 

40  .8 

0  .672 

2 

FT  Monmouth,  NJ  to  Palo  Alto,  CA 

2.81 

12.9 

4.57 

17.7 

19.9 

0.853 

3 

Guam  to  Honolulu,  Hawaii 

-1  .12 

-4.1 

3  .30 

7.4 

18.6 

0  .922 

4 

Guam  to  Kodiak,  Alaska 

-2.60 

-15.9 

3.48 

18.3 

21.1 

0  .902 

5 

Honolulu,  Hawaii  to  Kodiak,  Alaska 

-0  .09 

-5  .3 

3.53 

16.6 

18.2 

0  .352 

6 

Honolulu,  Hawaii  to  Washington,  DC 

4.52 

23.3 

5.37 

23.8 

22.6 

0.832 

7 

Davis,  CA  to  Honolulu,  Hawaii 

2.01 

9.9 

3.39 

11.1 

13.2 

0  .864 

8 

Palo  Alto,  CA  to  Fairbanks,  Alaska 

3.14 

16.1 

3.72 

16.7 

15.5 

0.742 

9 

Boulder,  00  to  Pt.  Barrow,  Alaska 

4.15 

21.7 

4.69 

22.2 

21  .4 

0  .754 

10 

Honolulu,  Hawaii  to  Yokohama,  Japan 

-1.77 

-9.4 

3.64 

11.0 

19.8 

0.895 

11 

Tarlac,  Philippines  to  Yokohama,  Japan 

-0  .59 

-1  .9 

2  .0  3 

5.4 

7.1 

0  .888 

12 

Tar lac,  Philippines  to  H.E.  Holt, 
Australia 

-0.10 

1.8 

4.36 

11.0 

16.8 

0.819 

13 

Guam  to  H.  E.  Holt,  Australia 

1.41 

5.8 

2.69 

6.3 

8.8 

0.907 

14 

Davis,  CA  to  Kodiak,  Alaska 

-0  .99 

-7.4 

5.39 

11.1 

28.4 

0  .665 

15 

Honolulu,  Hawaii  to  Corona,  CA 

0.21 

-4.9 

4.43 

18.4 

23.3 

0.844 

16 

Andoya,  Norway  to  Thessoloniki,  Greece 

-1  .53 

-13.6 

2.00 

15  .5 

19.0 

0  .9  75 

17 

Davis,  CA  to  La  Posta,  CA 

1.94 

19.3 

2.77 

21.7 

19.3 

0.710 

18 

Toulouse,  France  to  Neimakri,  Greece 

-4.74 

-49.1 

5.91 

50  .4 

64.1 

0  .649 

19 

Honolulu,  Hawaii  to  La  Posta,  CA 

-0.47 

-2.9 

2.34 

6.3 

15.3 

0.862 

20 

Coco  Solo,  Canal  Zone  to  Stockbridge, 

NY 

Andoya,  Norway  to  New  Delhi,  India 

0.17 

-0  .1 

2.97 

6  .4 

1 1  .4 

0.920 

21 

1.66 

10.1 

2.52 

10  .9 

12.7 

0.90  5 

22 

Palo  Alto,  CA  to  Thule,  Greenland 

2.41 

12.0 

3.55 

19.5 

20  .9 

0  .872 

23 

Toulouse,  France to  Keflavik,  Iceland 

0.51 

4.0 

2.68 

6.1 

16.7 

0.824 

24 

FT  Monmouth,  NJ  to  Aberdeen,  NY 

1  .06 

14.4 

1  .19 

14.8 

13.6 

0  .316 

25 

FT  Monmouth,  NJ  to  Camp  Drum,  NY 

0.56 

6.3 

0  .90 

7.7 

10  .4 

0.935 

Table  4.  ITSA-1  (universal  time  tape)  comparison  by  sounder  path. 
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I 


Corre¬ 

lation 


No. 

Transmission  Path 

Bias, 

MHz 

% 

rms, 

MHz 

error, 

% 

Magni¬ 
tude  ,  % 

Coeffi¬ 

cient 

1 

Guam  to  Yokohama,  Japan 

-3.07 

-17.5 

6.57 

18.9 

40  .4 

0  .668 

2 

FT  Monmouth,  NJ  to  Palo  Alto,  CA 

2.58 

10  .6 

4.66 

18.2 

21.6 

0.827 

3 

Guam  to  Honolulu,  Hawaii 

-1  .05 

-4.2 

3.34 

6.7 

18.8 

0  .906 

4 

Guam  to  Kodiak,  Alaska 

-1.92 

-1 1 .9 

3.34 

15.4 

19.8 

0.862 

5 

Honolulu,  Hawaii  to  Kodiak,  Alaska 

0  .50 

-2  .3 

4.17 

16.9 

19.5 

0  .771 

6 

Honolulu,  Hawaii  to  Washington,  DC 

4.38 

22.7 

5.17 

23.0 

22.2 

0.852 

7 

Davis,  CA  to  Honolulu,  Hawaii 

2  .47 

12.2 

3  .84 

13.6 

14.9 

0  .838 

8 

Palo  Alto,  CA  to  Fairbanks,  Alaska 

3.82 

20.2 

4.29 

20.3 

19.3 

0 .747 

9 

Boulder,  CO  to  Pt.  Barrow,  Alaska 

4.39 

22.9 

5.10 

23.6 

22.4 

0  .601 

10 

Honolulu,  Hawaii  to  Yokohama,  Japan 

-1.32 

-7.9 

3.42 

10.4 

18.1 

0.887 

11 

Tarlac,  Philippines  to  Yokohama,  Japan 

-0  .64 

-2  .0 

2  .04 

6.5 

6.9 

0  .932 

12 

Tarlac,  Philippines  to  H.E.  Holt, 

0.08 

2.4 

4.56 

10.5 

17.6 

0.784 

13 

Australia 

Guam  to  H.  E.  Holt,  Australia 

1.19 

5.2 

2.54 

6.2 

8.2 

0  .915 

14 

Davis,  CA  to  Kodiak,  Alaska 

-0  .96 

-7.0 

5.74 

10  .9 

30  .3 

0.631 

15 

Honolulu,  Hawaii  to  Corona,  CA 

0.68 

-3.6 

4.50 

19.6 

21.7 

0  .854 

16 

Andoya,  Norway  to  Thessoloniki,  Greece 

-1  .00 

-9  .6 

1  .45 

1 1  .6 

13.7 

0  .982 

17 

Davis,  CA  to  La  Posta,  CA 

2.00 

19.7 

2.75 

21  .6 

18.1 

0.713 

18 

Toulouse,  France  to  Neimakri,  Greece 

-4.30 

-44.0 

5.55 

45.4 

59.0 

0  .662 

19 

Honolulu,  Hawaii  to  La  Posta,  CA 

-0.41 

-2.8 

2.37 

7.8 

15.7 

0.853 

20 

Coco  Solo,  Canal  Zone  to  Stockbridge, 

NY 

1.17 

3.6 

3.40 

8.2 

10  .9 

0  .907 

21 

Andoya,  Norway  to  New  Delhi,  India 

2  .04 

13.1 

2.77 

13.8 

13  .4 

0  .90  8 

22 

Palo  Alto,  CA  to  Thule,  Greenland 

2.96 

16.0 

3.94 

21.9 

22  .4 

0  .879 

23 

Toulouse,  Franceto  Keflavik,  Iceland 

0  .72 

5.6 

2.95 

7.7 

18.0 

0  .792 

24 

FT  Monmouth,  NJ  to  Aberdeen,  NY 

1  .07 

14.2 

1.17 

14.3 

13.6 

0.820 

25 

FT  Monmouth,  NJ  to  Camp  Drum,  NY 

0.89 

11  .1 

1  .23 

1  1  .7 

14.1 

0  .893 

Table  5.  ITSA- 1  (local  time  tape)  comparison  by  sounder  patn. 
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1 


Program 

Bias 

MHz 

rms, 

MHz 

error, 

% 

Magni¬ 
tude  % 

Corre¬ 

lation 

Coeffi¬ 

cient 

Error 

of  the 
Estimate , 
MHz 

MINIMUF-3.5 

0.08 

0.6 

3.71 

3.6 

20.3 

0.866 

3.56 

HFMUFES  4 

-1  .49 

-7.2 

4.24 

8.3 

26.0 

0.871 

3.50 

ITSA-1  (Universal  Time  Tape) 

-0.06 

-0.3 

3.95 

3.5 

21.6 

0.850 

3.75 

ITSA-1  (Local  Time  Tape) 

0.16 

0.7 

4.04 

3.5 

21.5 

0  .840 

3.86 

Table  6. 

Overall 

cotqparison  results. 

Measurement  Method  Sample  Path  Hours  Percent  of  Sample 


NTSS-HFDR 

1416 

30.3 

NTSS-strip  chart 

2388 

51  .2 

Non-NTSS 

48 

1.0 

Granger  900  series 

744 

15.9 

Modified  C-3 

72 

1.5 

Table  7.  Number  of  samples  per  sounder  data  type. 


Figures  13  and  14  show  the  average  residual  (bias)  and  average  relative 
residual,  respectively,  as  a  function  of  data  type.  MINIMUF-3.5  has  a  near 
zero  bias  nuch  of  the  time  except  for  the  non-NTSS  data.  The  biases  in  the 
two  ITSA-1  programs  follow  each  other  very  closely  with  the  bias  being  smaller 
for  ITSA-1  with  universal  time  tape.  HFMUFES  4  predicts  high  by  more  than  10 
percent  for  81  .5  percent  of  the  sample  and  low  for  the  remaining  portion  of 
the  data. 

Figures  15  and  16  shew  the  rms  error  and  relative  rms  error,  respec¬ 
tively.  The  rms  error  for  MINIMUF-3.5  is  less  than  8  percent  for  all  the  data 
except  the  non-NTSS  data,  which  represents  only  1  percent  of  the  sanple.  For 
the  NTSS  data,  HFMUFES  4  has  its  highest  rms  error,  being  as  high  as  13 
percent.  For  the  NTSS  data,  the  two  ITSA-1  programs  fall  between  MINIMUF-3.5 
and  HFMUFES  4.  The  ITSA-1  program  is  shown  to  have  slightly  lower  rms  error. 

Figure  17  shows  the  magnitude  of  the  error.  This  figure  clearly  illus¬ 
trates  the  lower  magnitude  of  the  error  of  MINIMUF-3.5.  For  the  type  of  data 
for  which  HFMUFES  4  has  low  bias  and  rms  error,  its  magnitude  of  the  error  is 

also  lowest. 


34 


Ave ruie  resi'ludl  (bias)  as  a  function  of  data  type. 


Figure  14.  Average  relative  residual  (relative  bias)  as  a  function  of  data 


Figure  15.  Rms  error  in  MHz  as  a  function  of  data  type. 
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Figure  18  shows  the  correlation  coefficient  of  the  predicted  mjf  and 
observed  MOF  as  a  function  of  data  type.  It  indicates  the  generally  high 
correlation  of  all  four  programs  with  the  data  type  except  the  taodified  0  3 
data.  In  this  case  the  one  sample  per  hour  might  not  be  sufficient  for 
obtaining  a  good  monthly  median  at  each  hour. 


PATH  LENGTH 


Figures  19-24  show  the  distribution  of  MUF  prediction  error  as  a  function 
of  path  length.  Table  8  shows  the  percentage  of  the  sample  in  each  path 
length  range.  Figures  19  and  20  show  the  average  residual  and  average  rela¬ 
tive  residual,  respectively.  They  show  MINIMUF-3.5  to  have  the  smallest  bias 
in  general  for  path  lengths  at  or  lower  than  4000  tan.  Beyond  4000  km, 

MINIMUF-3.5  and  the  ITSA-1  programs  have  nearly  the  same  bias.  HFMUFES  4  has 
the  highest  bias  and  always  tends  to  predict  high.  Figures  21  and  22  show  the 
rms  error  and  the  relative  rms  error,  respectively.  There  is  a  peak  in  rms 
error  at  2000  km.  However,  this  range  is  represented  by  only  one  path:  the 
Toulouse,  France,  to  Neimakri,  Greece,  path  which  comprises  only  two  percent 
of  the  sample.  With  one  exception  Figure  22  shows  all  four  programs  to  have 
less  than  10  percent  rms  error  beyond  2000  km.  The  exception  occurs  for 

HFMUFES  4  in  the  range  4000  to  5000  km.  Figure  23  shows  that  the  average 
magnitude  of  the  error  as  a  function  of  range  is  generally  less  than  20 
percent  except  at  the  previously  highlighted  ranges;  1000-2000  km  and  4000- 

5000  km.  Figure  24  shows  the  correlation  between  the  predicted  MUF  and 

observed  MOF  at  ranges  beyond  1000  km,  MINIMUF-3.5  shows  correlation 
coefficients  greater  than  0.8;  whereas,  the  correlation  coefficients  for  the 
other  three  programs  increase  with  increasing  range  to  a  peak  of  approximately 
0.95  at  7000  km  where  a  sharp  decline  then  begins. 

Percentage  of  Sample  Length  Numbers  of  Hours 


L  « 

1000 

192 

4.1 

1000 

< 

L 

< 

2000 

96 

2.1 

2000 

< 

L 

< 

3000 

640 

13.7 

3000 

< 

L 

4000 

552 

11.8 

4000 

< 

L 

< 

5000 

1608 

34.4 

5000 

< 

L 

6000 

586 

12.6 

6000 

< 

L 

* 

7000 

754 

16.2 

7000 

< 

L 

< 

8000 

240 

5.1 

Table  8.  Percentage  of  sample  in  each  path  length  range. 
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Figure  19.  Average  residual  (bias)  as  a  function  of  path  length 
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Figure  22.  Relative  rms  error  as  a  function  of  path  length. 


PATH  ORIENTATION 


Figures  25-30  summarize  the  performance  of  the  four  programs  as  a  func¬ 
tion  of  path  orientation.  This  categorization  is  important  to  assure  that  the 
sunrise/sunset  reactions  are  correct  for  varying  degrees  of  path  illumina¬ 
tion.  The  north-south  (N-S)  paths  are  those  which  lie  nominally  within  ±  15® 
of  a  0°  or  180®  bearing.  The  east-west  (E-W)  paths  are  those  which  fall  nom¬ 
inally  within  +  15°  of  a  90®  or  270®  bearing.  The  paths  which  did  not  meet 
either  criterion  were  put  in  the  "other"  category.  The  percentage  of  the 
sample  in  each  category  is  indicated  in  Table  9.  Table  10  indicates  which 
paths  are  in  ea'-h  category. 

Path  Orientation  Number  of  Hours  Percentage  of  Sample 

North/South  1072  23.0 

East/West  1042  22.3 

Other  2554  54.7 

Table  9.  Percentage  of  sample  in  path  orientation  categories. 


Figures  25  and  26  illustrate  the  bias  in  the  four  programs.  All  the  pro¬ 
grams  have  a  higher  bias  for  paths  oriented  in  the  north/  south  direction. 
This  is  not  surprising,  considering  the  dynamics  of  the  abrupt  F-region 
changes  which  occur  when  the  entire  path  is  illuminate.-:  sullenly,  as  on  the  N- 
S  paths.  However,  in  the  case  of  MINIMUF-3.5  and  the  two  ITSA-1  programs  the 
amount  of  the  bias  is  small  enough  to  assume  consistent  results  irrespective 
of  path  orientation. 

Figures  27  and  28  illustrate  the  rms  error  and  relative  rms  error, 
respectively.  For  MINIMUF-3.5  the  rms  error  ranges  between  about  4  percent 
(3.3  MHz)  and  6  percent  (4  MHz);  whereas,  for  HFMUFES  4  it  ranges  between  7.5 
percent  (3.5  MHz)  and  12.5  percent  (4.7  MHz). 

Figure  29  shows  the  average  magnitude  of  the  error.  For  MINIMUF-3.5,  the 
program  with  lowest  error,  it  ranges  between  about  19  percent  and  22  percent. 
For  the  worst  program,  HFMUFES  4,  the  error  ranges  from  about  22.5  percent  to 
28  percent. 
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Latitude  of  Geographic 


NO. 

Transmission  Path 

Orientation 

Control  Points 

Region 

1 

Guam  to  Yokohama,  Japan 

N-S 

LO 

B 

2 

FT  Monmouth,  NJ  to  Palo  Alto,  CA 

E-W 

M 

A 

3 

Guam  to  Honolulu,  Hawaii 

E-W 

LO 

B 

4 

Guam  to  Kodiak,  Alaska 

Other 

M 

B 

5 

Honolulu,  Hawaii  to  Kodiak,  Alaska 

N-S 

M 

B 

6 

Honolulu,  Hawaii  to  Washington,  DC 

Othe  r 

M 

C 

7 

Davis,  CA  to  Honolulu,  Hawaii 

E-W 

M 

B 

8 

Palo  Alto,  CA  to  Fairbanks,  Alaska 

Other 

H 

C 

9 

Boulder,  CO  to  Pt.  Barrow,  Alaska 

Other 

H 

A 

10 

Honolulu,  Hawaii  to  Yokohama,  Japan 

Other 

M 

B 

11 

Tarlac,  Philippines  to  Yokohama,  Japan 

Other 

LO 

B 

12 

Tarlac,  Philippines  to  H.E.  Holt, 
Australia 

CO 

1 

z 

TE 

C 

13 

Guam  to  H.  E.  Holt,  Australia 

Other 

TE 

C 

14 

Davis,  CA  to  Kodiak,  Alaska 

Other 

M 

B 

15 

Honolulu,  Hawaii  to  Corona,  CA 

Other 

M 

B 

16 

Andoya,  Norway  to  Thessoloniki,  Greece 

N-S 

H 

A 

17 

Davis,  CA  to  La  Posta,  CA 

Other 

M 

A 

18 

Toulouse,  France  to  Neimakri,  Greece 

E-W 

M 

C 

19 

Honolulu,  Hawaii  to  La  Posta,  CA 

Other 

M 

B 

20 

Coco  Solo,  Canal  Zone  to  Stockbr i ige , 

NY 

N-S 

M 

B 

21 

Andoya,  Norway  to  New  Delhi,  India 

E-W 

H 

A 

22 

Palo  Alto,  CA  to  Thule,  Greenland 

N-S 

TA 

Pi 

23 

Toulouse,  France  to  Keflavik,  Iceland 

Other 

H 

C 

24 

FT  Monmouth,  NJ  to  Aberdeen,  N’Y 

Other 

M 

A 

25 

FT  Monmouth,  NJ  to  Camp  Drum,  NY 

N-S 

M 

A 

TE  =  Transequatorial 

E-W 

=  East/west 

l =  Low  latitude 

N-S 

=  North/south 

M  =  Mid-latitude 

A  = 

Continental 

H  =  Hign  latitude 

B  = 

Ocean 

I A  =  Transauroral 

C  = 

Combined  land/ocean 

Table  10.  Additional  path  characteristics. 
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Figure  25.  Average  residual  (bias)  as  a  function  of  orientation 


function  of 


Figure  27.  Rms  error  as  a  function  of  orientation. 


Figure  28.  Relative  rus  error  as  a  function  of  orientation. 
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tror  (average  absolute  relative  residual)  as 


In  Figure  30,  HFMUFES  4  is  clearly  shown  to  have  the  highest  correlation 


with  path  orientation.  All  of  the  programs  have  correlation  coefficients 
greater  than  0.8. 

SJSASON/MONTH 

Figures  31-36  summarize  the  performance  of  the  four  programs  as  a  func¬ 
tion  of  season  and  Figures  37-42  provide  additional  detail  as  a  function  of 
month.  Here  the  seasons  are  defined  as:  (1)  winter,  November  through  Febru¬ 
ary;  (2)  spring,  March  and  April;  (3)  summer,  May  through  August;  and  (4) 
autumn,  September  and  October. 

During  the  winter,  MINIMUF  predicts  0.5  Ml,z  (3.5  percent)  low  as  shown  in 
Figures  31-32.  The  other  programs  predict  high.  The  worst  was  HFMUFES  4, 
which  predicted  2  MHz  (11  percent)  nigh.  The  months  primarily  affecting  these 
results  for  MINIMUF-3.5  were  November  and  December,  for  which  it  was  low  by  as 
much  as  6  percent. 

During  the  summer  months,  HFMUFES  4  is  shown  to  predict  quite  close; 
whereas  MINIMUF-3.5  predicts  high  by  as  much  as  0.8  MHz  (2  percent).  During 
these  months  the  two  ITSA-1  programs  performed  at  their  worst.  ITSA-1  with 
universal  time  tape  is  low  by  1.3  MHz  (7  percent),  and  ITSA-1  with  local  time 

tape  is  low  by  1.7  MHz  1  percent).  The  large  scale  programs  have  the  most 

trouble  predicting  June  accurately;  whereas  MINIMUF-3.5  has  the  most  trouble 
predicting  August  accurately. 

During  the  equinox  months,  MINIMUF-3.5  has  most  difficulty  predicting 
October  accurately,  where  it  is  low  by  1.9  MHz  (9  percent,;  whereas  HFMUFES  4 
had  its  chief  difficulty  predicting  March,  where  it  is  hi  hr.  by  3.5  MHz  (17 
percent ) . 

The  rms  error,  as  illustrated  .n  Figures  33-34  and  39-40,  shows  somewhat 
the  same  pattern  for  each  program.  MINIMUF-3.5  has  its  largest  rms  erior 

during  fall  (October)  with  a  value  of  roughly  4.1  MHZ  (7  percent).  HFMUFES  4 

has  its  highest  value  during  spring  (March)  with  a  value  of  nearly  5  MHz  16.5 
percent) . 

The  magnitude  of  the  error  is  shown  in  Figure  35  to  be  rat.er  constant  at 
about  20  percent  for  MINIMUF-3.5.  Whereas  for  HFMUFES  4  it  is  above  30  per- 
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Figure  31.  Average  residual  (bias)  as  a  function  of  season. 
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cent  during  winter  and  spring,  dropping  be  1  uw  it 
rising  to  about  23  percent  in  the  rail. 

rigute  3o  shows  HKMUFES  4  to  nave  tnc  n. 
a  function  of  season.  Its  hignest  value  is  abi'V 
lowest  value  is  about  "j  .  ns 5  during  the  sann  :  . 
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Figure  48.  Correlation  coefficients  as  a  function  of  geomagnetic  latitude 
location  of  control  points. 
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MINIMUF-3.5  are  not  much  different.  However,  at  low  latitude  the  ITSA- 1 
programs  have  as  much  difficulty  as  does  HFMUFES  4. 

Figure  46  shows  that  the  relative  rms  error  increases  for  all  four 
programs  as  the  geomagnetic  latitude  of  the  control  points  increase.  What  is 
not  expected  is  the  peak  for  HFMUFES  4  occurring  at  mid- latitude.  In  this 
latitude  region  one  would  expect  it  to  perform  at  its  best. 

Figure  47  shows  that  the  average  magnitude  of  the  error  for  both  HFMUFES 
4  and  MINIMUF-3.5  peak  at  mid-latitude  with  HFMUFES  4  has  a  value  over  30 
percent.  MINIMUF-3.5  generally  has  values  less  than  20  percent  at  all  geomag¬ 
netic  regions. 

Again  Figure  48  shows  the  superior  correlation  between  predicted  MUF 
values  by  HFMUFES  4  and  the  observed  values  except  for  the  transaurora  1  region 
where  MINIMUF-3.5  has  the  highest  values. 

SOLAR  SUNSPOT  NUMBER  (SSN) 

A  major  consideration  in  MUF  prediction  is  the  ability  of  a  model  to  deal 
with  different  phases  of  the  solar  sunspot  cycle.  Ideally,  it  should  produce 
consistent  results  for  SSN  values  between  one  and  150.  The  data  were  sub¬ 
divided  into  four  sunspot  number  categories  (actually  five,  but  the  fifth  con¬ 
tained  no  data  points).  Table  12  gives  the  percentage  of  the  sample  in  each 
category. 

Sunspot  Number 


(Cycle  Phase) 

Sample,  Path  hours 

*  of  Sai 

10-30 

( minimum) 

1728 

37.0 

31-60 

(rise  and  decline) 

2  40 

5.1 

61-90 

(near  maximum) 

551 

1 1  .8 

91-120  (maximum) 

210  1 

45.0 

121-150  (high  maximum) 

0 

0  .0 

Table  12.  Percentage  of  sample  in  each  SSN  category. 

Fi pares  49-54  summarize  the  prediction  performance  as  a  function  of  S-  ». 
Figures  49-53  show  the  same  poor  per forma nee  of  HFMUFES  due  to  its  tendency  *o 
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Figure  49.  Average  residual  (bias)  as  a  function  of  sunspot  number. 


in  MHz  as  a  function  of  sunspot  number. 


us  a  function  of  sunspot  nuir.bo 


predict  high.  The  remaining  results  show  the  somewhat  superior  performance  of 

MINIMUF-3.5.  Figures  49  and  50  show  a  spread  in  bias  between  a  SSN  of  60  to 
90.  Figure  51  shows  that  the  rms  error  increases  with  increasing  SSN,  but 
Figure  52  shows  that  the  relative  rms  error  decreases  with  high  SSN.  This 
probably  is  due  to  increasing  MOFs  at  high  SSN,  and  hence,  decreasing  relative 
residuals  at  high  SSNs.  The  relative  rms  error  seems  to  peak  at  SSN  between 
oO  and  90,  being  at  most  d  percent  for  MlNIMLF-3 . 5 .  Figure  54  shows  again  the 
superior  correlation  between  HHMUFES  4  predicted  data  and  observed  MQF  data. 


DIURNAL  TRENDS 

One  of  the  most  important  variations  in  path  MOF  is  its  diurnal  varia¬ 
tion.  This  section  describes  the  accuracy  of  the  programs  as  a  function  cf 
time  of  day.  To  do  this,  the  entire  data  set  was  converted  to  local  path  time 
;i.e.,  the  local  time  at  the  path  midpoint).  Figures  55-60  show  the  results 
of  the  comparison  of  the  four  programs  as  a  function  of  midpath  local  time. 

Figures  55  and  56  show  the  average  residual  and  the  average  re!a„-*e 
residual,  respectively.  HFMUFES  4  predicts  high  at  a.l  hours  having  '-ts 
largest  bias  of  2.7  MHz  ',11  percent)  at  1600  LT.  The  bias  of  the  three  1 1:  >■ 

scale  programs  show  a  strong  correlation  as  a  fur  :t  ict  of  local  time, 

bias  of  MINIMUF-3.5  has  a  strong  iiurnal  variation.  -tatting  at  100  !.T  '  m 

average  residual  is  1.3  below  the  observed  value.  The  tea*  ie  creases  v 
cert,  at  0700  LT,  but  increases  again  to  1  MHz  low  i*.  .  60  -  LT.  :  •  » - 

readily  increases  then  .-.s  local  time  met  eases  t  -  a  value  of  1 .  "5  Miit  i* 

'  ,11>  LI.  ’  t  the:’.  m  ,  r-.-.j  -es  with  local  time  •  ‘  vdz  1  -  it  140  1.1  . 


c  a  • 


Fig-it -s  57  and  56  =  h.-w  the  rs  rr.i-  and  :  ;  at  ive  rms ,  respect  *.v-  lv. 
cost  notable  feature  u.  these  ft  uvu  ’  "s  -rr  r  ■  tne  M.NiMi!-!.  ,  . 
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Figure  55.  Average  residua L  (bias)  as  a  function  of  local  time 
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transits  n  period.  l  ie  iragnitude  of  the  error  for  F.FMUt  ~ 4  is  again  the 
highest  of  the  four. 

Figure  60  shews  the  correlation  coefficient  between  the  predicted  and 
observed  values.  The  correlation  is  generally  high  with  HFMUFES  having  the 
highest  values. 

GEOGRAPHICAL  KBS IONS 

The  last  variation  to  be  considered  was  the  effect  of  different  geograph¬ 
ical  regions  on  performance  of  the  programs.  The  subdivision  chosen  was  paths 
that  were  either  entirely  over  land  (continental) ,  entirely  over  ocean  (ocean) 
or  partly  over  land  and  partly  over  ocean  (other).  This  division  was  chosen 
partly  because  of  the  sparsity  of  data  in  ocean  areas  to  develop  the  numerical 
maps  of  ionospheric  coefficients  and  because  oblique  sounder  data  over  the 
ocean  areas  were  used  to  calibrate  MINIMUF-3.5.  Table  13  indicates  the 
percentage  of  the  sample  in  each  geographic  area.  Note  the  dominance  of  data 
from  paths  over  the  ocean  and  the  small  portion  of  data  entirely  over  land. 
Figures  61-66  illustrate  the  performance  as  a  function  of  geographic  region. 


Geographic  Reqior. 

Path  Hours 

Percentage  of  Sample 

Continental 

624 

13.4 

Ocean 

2738 

58.7 

Other 

1306 

28.0 

Table  13.  Percentage  of  sample  in  geographic  regions. 

Figures  61  and  62  show  the  average  residual  and  average  relative  resid¬ 
ual,  respectively.  Note  first  that  HFMUFES  4  performs  'worst  in  the  ocean 
areas.  In  fact,  its  performance  in  the  other  regions  is  actually  better  than 
the  other  programs.  In  the  ocean  areas  MINIMUF-3.5  produces  the  best  results. 

Figures  63  and  64  show  the  rms  error  and  relative  rms  error,  respective¬ 
ly.  Again  HFMUFES  4  has  its  highest  value  (nearly  4.7  MHz)  over  ocean  areas. 
By  contrast,  over  land  its  rms  error  drops  to  2.5  MHz.  As  expected  MINIMUF- 
3.5  has  its  lowest  relative  rms  error  over  ocean  paths. 

Figure  65  shows  the  average  magnitude  of  the  error  of  the  four  programs. 
In  this  figure,  as  was  the  case  in  the  previous  four  figures,  HFMUFES- 4  is  the 
worst  program  over  ocean  paths  and  the  best  over  land  bodies. 
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Figure  61.  Average  residual  (bias)  as  a  function  of  geographic  region 


Figure  61#  Urns  error  In  MHz  aw  a  function  of  geographic  region. 


Figure  64.  Relative  rms  error  as  a  function  of  geographic  region. 


Figure  65 .  (“latitude  of  error  (average  absolute  relative  r'Si du a  1 )  as 
function  of  geographic  region. 


Figure  66  shows  the  correlation  coefficient  of  the  four  programs  as  a 
function  of  geographical  region.  All  the  programs  have  values  greater  than 
0.8,  with  HFMUFES  4  being  superior. 


DISCUSSION  OF  RESULTS 

This  report  presented  the  accuracy  of  predicted  MUFs  by  four  prediction 
programs.  This  was  done  by  comparing  predicted  median  MUFs  to  corresponding 
oblique  sounder  median  MOFs.  When  compared  overall,  MINIMUF-3.5  and  the  two 
ITSA-1  versions  were  separated  only  a  few  tenths  of  a  percent  in  bias  and  rms 
error  and  by  just  one  percent  in  magnitude  of  the  error.  However,  HFMUFES  4 
clearly  had  the  largest  bias  (7.2  percent  high),  rms  error  (8.3  percent)  and 
magnitude  of  error  (26.09  percent).  All  the  programs  had  difficulty  predict¬ 
ing  high  latitude  and  transauroral  paths  accurately.  For  path  lengths  between 
4300  and  5000  km  HFMUFES  4  had  difficulty  predicting  accurately.  MINIMUF-3.5 
had  difficulty  predicting  accurately  during  the  morning  and  evening  transition 
hours.  But  perhaps  the  most  surprising  result  was  that  HFMUFES  4  was  tne  most 
accurate  over  land  paths  and  the  least  accurate  over  ocean  paths. 

EXPLANATION  FOR  HFMUFES  4’ S  PERFORMANCE 

The  performance  of  HFMUFES  4  was  disappointing  except  over  land.  As  it 
is  is  the  most  recent  of  the  ITS-78  line  of  HF  prediction  programs,  one  would 
expect  its  results  to  be  most  accurate  overall.  There  are  two  possible 
explanations  for  the  inaccuracy  m  HFMUFES  4  XUF  prediction.  These  are:  (1) 
bias  in  the  numerical  map  of  f0F2  coefficients  and  (2)  method  of  determining 
the  k'  sec  j>  factor  in  the  MUF  calculation. 

Bias  in  fQF2  Coefficients 

One  of  the  major  differences  in  the  three  large  scale  programs  is  the 
numerical  map  of  fQF2  coefficients.  In  earlier  versions  of  the  ITSA-1 
program,  fQF2  is  a  function  of  local  time. 19  Later  versions  of  the  program 
calculated  fQF2  as  a  function  of  universal  time  and  so-cal^d  modified 
magnetic  dip.2®  In  HFMUFES  4  the  numerical  coefficients  of  the  F2  layer 
critical  frequency  were  revised  to  include  solar  cycle  and  seasonal  variations 
of  f0F2.24 


98 


T:ie  numerical  napping  procedures  involved  two  principal  analyses  of  the 
vertical  ionosonde  data.  First,  a  so-called  "screen  analysis”  was  made  of 
available  A-  and  B-data  to  produce  approximate  values  of  the  ionospheric 
characteristic  (called  C-data)  at  carefully  chosen  locations  ("screen  points") 
in  large  regions,  such  as  oceans,  where  no  ionosonde  stations  were  available. 
The  C-data  were  then  combined  with  the  original  A-  and  B-data  in  forming  a 
second  (and  final)  analysis.  A-data  referred  to  measurements  at  stations 
taken  during  the  actual  month  in  question.  B-data  referred  to  values  of  the 
characteristics  obtained  from  interpolation  or  extrapolation  in  time  at  a 
station  which  did  not  report  data  for  the  specific  month  in  question,  but  was 
active  for  a  period  of  time  before  or  after.  B-data  were  used  to  fill  gaps 
where  A-data  were  not  available. 

In  the  oblique  incidence  sounder  data  used  to  compare  the  programs,  58.7 
percent  of  the  sample  was  for  paths  entirely  over  the  ocean.  Hence,  any  bias 
in  the  numerical  maps  due  to  the  use  of  C-data,  which  is  also  primarily  from 
ocean  areas,  would  be  reflected  in  the  bias  in  the  MIT  predictions  and  the 
bias  in  fQF2  would  be  accentuated  by  k*  sec  1  factor  ir.  the  MUF  calculation. 
In  the  case  of  KFMUFE'S  4,  the  numerical  map  is  more  complex  than  in  ITSA-1. 
Hence,  its  bias  may  be  larger  chan  the  other  two  programs  over  the  ocean. 

As  the  C-data  only  approximates  what  may  have  beer,  recorded  by  vertical 

ionosondes  over  ocean  areas,  the  only  clue  to  the  bias  m  the  numerical  map 

24 

representing  fQF2  ir.  HFMUFES  4  is  that  published  fc:  ere  A-  and  B-data. 
Table  14  gives  both  the  average  deviation  (bias)  and  average  relative 
deviation  of  the  predicted  f 0F2  from  the  observed  fQFi  : .'r  each  month  of  1964 
and  1967,  periods  of  low  and  moderately  high  solar  activity.  The  number  of 
stations  with  available  observed  data  varied  from  month  to  month  between  70 
and  100.  Bias  on  the  negative  side  (predictions  too  large)  appears  in  1964 
and  the  latter  part  of  1967  and  a  large  positive  bias  occur*  in  the  first  part 
of  1967. 
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Year 

Month 

R 

Average 

deviation 

(MHz) 

Average  relative 
deviation 

(%) 

1964 

1 

20 

-0.36 

-9.82 

2 

18 

-0  .30 

-7.66 

3 

15 

0.03 

0.19 

4 

13 

0.05 

-1  .76 

5 

11 

-0.16 

-4.49 

6 

10 

-0  .0  8 

-3.34 

7 

10 

-0  .20 

-5.63 

8 

10 

0  .01 

-0  .50 

9 

10 

0.07 

1  .89 

10 

10 

0  .02 

0  .44 

11 

10 

-0.03 

-1.52 

12 

11 

0  .11 

0  .77 

1967 

1 

75 

0.19 

0.76 

2 

79 

0  .43 

4.50 

3 

82 

0.83 

10.03 

4 

85 

0  .55 

6.39 

5 

87 

0.20 

1  .29 

6 

91 

-0  .06 

-2  .04 

7 

94 

0.05 

-0.22 

8 

95 

0.14 

1  .02 

9 

95 

-0.05 

-2.30 

10 

95 

0  .08 

0  .63 

1  1 

97 

-0.32 

-7.47 

12 

101 

-0  .15 

-4.50 

Table  14.  Median  deviations  from  numerical  map  predictions  of  fQF2  FES 

4  (analysis  in  running  average  sunspot  number  R) . 

Some  examples  of  bias  in  the  predicted  values  of  fQF2  in  HF'S.'fZ;  4  for 
stations  not  used  in  forming  the  numerical  map  are  given  ir.  T.t.e  15. 
Tbilisi,  Khabarovsk  and  Cape  Zevgari  are  at  mid-latitude;  and  Port  Moresby, 
Cocos  Island  and  Ouagadougou  are  at  low  latitude.  First  the  table  s  :*s  the 
bias  to  be  generally  negative  (predicted  values  too  high) .  Then  the  bias  at 
low  latitude  is  higher,  being  as  large  as  -26.7  percent. 
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Year 

Month 

Tbilisi 

Khabarovsk 

1964 

1 

-10.0 

-1.4 

3 

2.7 

3.4 

5 

-3.2 

-2.4 

7 

-6.4 

-6.1 

9 

2.3 

10.7 

11 

1  .5 

4.9 

1967 

1 

-1.9 

3 

9.0 

5 

1.2 

7 

-2.8 

9 

0.5 

11 

-15.9 

Cape 

Port 

Cocos 

Zevqari 

Moresby 

Island 

Ouaqadougoi 

-7.2 

-14.0 

-7.2 

-21 .0 

-12.8 

-13.0 

-15.3 

-13.0 

-14.7 

-7.8 

-6.4 

-7.0 

-9.6 

3.7 

-0.1 

-4.6 

2.2 

9.1 

7.6 

0  .6 

2.7 

0.7 

-10.1 

0.1 

2.6 

-9.5 

-26.7 

-4.0 

-6.5 

-9.1 

-5.0 

-5.2 

-4.7 

Tbilisi  (41. 7N,  44. 8E),  Khabarusk  (48. 5N,  135. IE),  Cape  Zevgari  (34. 6N,  32. 9E), 
Port  Moresby  (9.4S,  147. IE),  Cocos  Island  (12. 2S,  96. 8E),  Ouagdougou  (12. 9N, 

2.0W) 


Table  15.  Median  relative  deviations  in  percent  from  numerical  map  predictions 

of  f  F2  in  HFMUFES  4  for  several  stations, 
o 


VKRTICAL-TO-OHLIQUE  TRANSFORMATION 

Since  the  basic  data  available  on  a  world-wide  basis  are  obtained  from 
vertical  ionosondes,  a  transmission  must  be  effected  in  the  application  to  an 
oblique  path.  The  determination  of  oblique  propagation  characteristics  from 
vertical  data  is  relatively  simple  in  the  case  of  a  plane  stratified  iono¬ 
sphere  and  in  the  absence  of  the  earth's  magnetic  field.  Considerable  modifi¬ 
cation  is  brought  about  by  the  inclusion  of  ionosphere  curvature,  electronic 
collisions  and  the  magnetic  field,  and  the  general  theory  is  complicated. 

The  relationship  between  the  frequency  f 0j)  of  the  curve  incident 
obliquely  on  a  flat  layer  and  the  equivalent  vertical  frequency  fy  is 


f  ,  =  f  sec  $  (8) 
ob  v 

where  $  is  the  angle  between  the  vertical  and  the  oblique  ray  at  the  bottom  of 
the  layer.  Equation  (8)  is  known  as  the  secant  law.  It  shows  that  a  given 
ionospheric  layer  can  reflect  higher  frequencies  as  the  obliquity  of  the  ray 
paths  increase.  For  a  flat  ionosphere  the  MUF  is  given  by 
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(9) 


MUF  =  f  sec  t> 
c 

where  fc  is  the  critical  frequency. 

For  a  curved  ionosphere  the  above  relationships  must  be  corrected  to 

f  =  f  k  sec  <p  ( 10 ) 

ob  v 

where  k  sec  $  is  often  referred  to  as  secant  $  (corrected).  The  MUF  is  given 
by 


MUF  =  fc  k  sec  4>.  (ID 

In  the  large  scale  prediction  programs,  it  is  assumed  that  the  two  layers  (E 
and  F2)  can  be  represented  by  parabolic  layers.  For  the  parabolic  layer 
assumption,  k  sec  <P  is  a  function  of  h^,  height  of  maximum  electron  density  of 
the  layer;  of  ym,  the  semi-thickness  of  layer;  of  the  critical  frequency,  fc 
and  of  the  range  D.  The  variation  with  distance  of  the  k  sec  $  factor  for  a 
parabolic  layer  of  ym/hQ  =  0.4  is  reproduced  in  Figure  67.  The  curves  are 
parametric  in  height  of  maximum  electron  density  hm  (=  hQ  +  y^) .  For  a  height 
of  200  km,  k  sec  $  reaches  4.0,  corresponding  to  the  tangential  ray,  at  a 
ground  range  of  slightly  over  4000  km.  Whereas,  for  a  h^  of  400  km,  k  sec  $ 
reaches  a  maximum  value  of  just  under  3.0  with  a  ground  range  in  excess  of 
6000  km.  The  variation  of  ground  range  with  distance  for  various  small  angles 
of  elevation  (0°,  1°  ,  3°  and  5°)  are  also  shown.  For  these  angles  the 

increase  in  k  sec  $  as  a  function  of  range  is  small. 

When  the  critical  frequency  fc  is  multiplied  by  k  sec  $  in  Equation  (11), 
additional  bias  in  the  predicted  MUF  is  introduced.  This  increase  in  bias 
becomes  greater  with  increasing  range  D  and  will  be  higher  for  layers  with  low 
height  of  maximum  electron  density  For  path  lengths  in  the  range  4000  to 

5000  km,  the  bias  in  the  critical  fc  will  be  multiplied  by  numbers  varying 

from  2.6  to  4.0.  The  effect  on  the  error  in  predicted  MUF  due  to  the  k  sec  ♦ 

factor  is  non-linear;  whereas,  the  effect  due  to  bias  in  critical  frequency  is 
linear. 
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Figure  67.  k  sec  $  factor  versus  distance  for  different  heights  of  the 
maximum  of  the  layer  . 

If  the  parameters  hQ  (bottom  of  the  layer)  and  ym  are  known,  together 
with  the  critical  frequency  fc,  it  is  possible  to  determine  k  sec  $  and, 
hence,  MUF  by  iteration  of  the  formula  provided  by  Appleton  and  Beynon. 40-41 
The  procedures  for  this  iteration  are  different  in  HFMUFES  4  and  in  IT5A-1. 
In  addition,  different  numerical  maps  for  fQF2,  the  parameters  h0  and  ym  are 
determined  differently  in  the  three  programs. 


APPLICATION  OF  LINEAR  REGRESSION 

Regression  analysis  can  be  used  to  establish  the  relationship  between 

variables.  Regression  analysis  was  used  to  determine  whether  a  linear  rela¬ 
tionship  exists  between  the  predicted  and  observed  MUFs.  A  linear  relation¬ 
ship  would  establish  that  the  error  in  the  predicted  MUF  (see  Equation  (11)) 
was  due  mainly  to  the  bias  in  the  critical  frequency.  A  non-linear  relation¬ 
ship  would  indicate  that  the  error  in  the  predicted  MUF  was  mainly  due  to  the 
k  sec  r  factor  in  the  MUF  calculation. 

Regression  analysis  of  the  predicted  MJFs  on  the  observed  MOFs  was  first 
calculated  as  a  function  of  range.  Figure  68  gives  the  standard  error  of  the 
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estimate  as  a  £ unction  of  distance  for  each  of  the  four  proyiams.  The  corres¬ 
ponding  figure  without  regression  is  Figure  21.  There  is  a  considerable 
lowering  of  rms  error  in  all  four  programs.  The  largest  change  occurs  for  the 
three  large  scale  programs  in  the  1000  to  5000  km  range.  The  largest  change 
occurs  for  HFMUFES  4.  Under  linear  regression  it  performs  better  than  the  two 
ITSA-1  programs  and  is  the  best  program  beyond  4000  km. 

After  having  removed  the  linear  errors  in  predicting  MUF  (the  bias  in  the 
critical  frequency),  Figure  68  shows  the  remaining  non-linear  errors.  Note 
first  the  remarkable  similarity  between  all  three  large  scale  programs  in  the 
first  3000  km.  Figure  68  also  shows  a  remarkable  similarity  to  Figure  67,  the 
figure  showing  the  k  sec  $  factor  as  a  function  of  range.  There  is  the  same 
large  rise  in  the  first  3000  kr  and  the  same  leveling  off  in  the  next  1000  km. 
Then  there  is  a  decrease  ir.  error  due  to  a  change  in  mode  and  a  reduction  in 
the  corresponding  k  sec  factor.  After  6000  km  there  is  a  new  increase  in 
error  due  to  an  increasing  k  sec  $  factor  for  the  second  hop  mode.  However, 
MINIMUF-3.5  does  net  bsvtt  '  corresponding  decrease  in  error  at  6000  km  as  the 
other  three  programs.  This  might  indicate  an  error  in  its  calculation  of  its 
k  sec  <(>  factor  (M  factor  as  it  is  called  in  its  terminology).  Figure  69  shows 
the  standard  rms  error  for  HFMUFES  4  with  and  without  regression,  showing 
clearly  a  linear  relationship  between  the  predicted  MUF  and  observed  MOFs. 

Figure  70  shows  the  standard  error  of  the  estimate  of  linear  regression 
as  a  function  of  geomagnetic  latitude  of  control  points.  The  corresponding 
figure  without  regression  is  Figure  45.  Here  again,  note  the  reduction  in  rms 
error  for  all  four  prediction  programs.  In  particular,  note  that  HFMUFES  4 
now  performs  better  than  the  other  two  large  size  programs.  Figure  71  shows 
the  improvement  of  HFMUFES  4  under  linear  regression.  At  mid-latitude  there 
is  nearly  a  1  MHz  in$>rovement  in  rms  error.  Figure  72  shows  the  improvement 
in  rms  error  for  MINIMUF-3.5  with  linear  regression.  The  improvement  occurs 
mainly  for  transequatorial,  high  latitude  and  transauroral  paths. 

The  results  of  linear  regression  as  a  function  of  mid-path  local  time  was 
examined  next.  The  corresponding  figure  without  regression  is  Figure  57. 
There  is  some  improvement  shown  in  Figure  73  in  the  performance  of  the  large 
computer  programs  with  the  largest  occurring  during  the  daytime.  HFMUFES  4 
was  shown  to  have  the  best  performance  under  regression  of  the  three  large 
programs.  Figure  74  shows  the  improvement  under  linear  regression  for  HFMUFES 
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Figure  73-  Standard  error  of  estimate  as  a  function  of  local  time 
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During  the  daytime,  the  improvement  is  more  than  1  MHz.  Figure  75  shows 
the  results  of  linear  regression  applied  to  MIN1MUF-3.5.  Note  that  an 
improvement  is  made  in  its  performance  during  the  evening  transition  hours, 
but  that  during  the  morning  transition  hours  there  is  no  improvement.  This 
non-linear  error  indicates  that  the  k  sec  $  factor  part  of  the  MINIMUF-3.5  MUF 
calculation  needs  further  improvement. 

Finally,  linear  regression  was  applied  as  a  function  of  geographic 
region.  The  results  are  shown  in  Figure  76.  The  corresponding  results 
without  regression  are  displayed  in  Figure  63.  For  MINIMUF-3.5  the  biggest 
improvement  is  for  continental  paths  and  paths  that  are  a  combination  of  land 
and  ocean.  As  MINIMUF-3.5  was  optimized  for  ocean  paths,  little  improvement 
is  seen  there.  The  other  three  programs  show  an  improvement  in  all  regions 
with  the  improvement  for  HFMUFES  4  being  the  greatest.  Figure  77  coiqpares  the 
results  of  HFMUFES  rms  error  as  a  function  of  geographic  region  with  and  with¬ 
out  regression.  Particularly,  it  shows  a  vast  improvement  over  ocean  areas 
(more  than  1  MHz  improvement).  This  indicates  that  the  error  in  HFMUFES  is 
primarily  due  to  the  bias  in  the  critical  frequency  over  ocean  area. 
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Figure  76.  Standari  error  of  estimate  as  a  function  of  geographic  region. 
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CONCLUSIONS 


For  the  particular  oblique  path  studies  and  conditions  assumed,  the 
following  conclusions  about  the  accuracy  of  HF  MUF  prediction  can  be  stated: 

1.  MINIMUF-3.5  appears  most  accurate  overall 

2.  The  version  of  ITSA-1  with  ionospheric  characteristics  mapped  in 
universal  time  is  slightly  more  accurate  than  ITSA-1  with  ionospheric  charac¬ 
teristics  mapped  ir.  local  time 

3.  MINIi'flJF-3.5  had  difficulty  predicting  accurately  during  the  sunrise 
and  sunset  transition  hours  and  for  path  lengths  5000  to  7000  km 

4.  Except  for  land  paths,  the  performance  of  HFMUFES  4  was  disappointing 

5.  HFMUFES  4  had  difficulty  predicting  MUFs  accurately  for  paths  over 
ocean  areas  and  for  paths  with  lengths  between  4000  and  5000  tan 

6.  All  of  the  programs  had  difficulty  predicting  MUFs  accurately  at  high 
latitudes 

The  inaccuracy  of  HFMUFES  4  is  due  in  part  to  the  existing  bias  in  the 
numerical  map  of  f ^c2  over  ocean  areas.  Except  for  a  few  lonosonde  stations 
on  islands  like  Hawaii,  there  were  no  real  ocean  area  vertical  ionosonde  data 
used  to  generate  the  numerical  map.  Instead,  ocean  area  data  were  generate! 
by  interpolating  between  existing  land  stations. 

When  the  critical  frequency  f  is  multiplied  by  the  so-called  ”k  sec  1 
factor,"  additional  -irror  in  the  predicted  MUF  is  introduced.  This  increase 
in  error  rises  with  increasing  range  and  is  higher  for  layers  with  low  height 
of  maximum  electron,  density. 

The  use  of  .near  regression  analysis  demonstrated  that  the  source  of 
error  in  HFMUFES  1  predicted  MUFs  (and  to  some  extent  ITSA-1  predicted  MUF) 
was  due  to  the  bias  in  the  input  critical  frequency  data,  not  inaccuracies  in 
the  k  sec  $  factor.  This  occurs  because  the  bias  in  critical  frequency- 
affects  the  MUF  linearly;  whereas,  the  k  sec  $  factor  affects  the  MUF  non- 
linearly.  In  the  case  of  HFMUFES  4,  the  bias  in  critical  frequency  is 
accentuated  by  tne  *  sec  ♦  factor. 

In  the  case  of  MINIMUF-3.5,  linear  regression  showed  that  the  errors  in 
predicted  MUFs  during  the  sunrise  transition  period  and  for  path  lengths  5000 
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to  7000  km  were  non-linear  effects.  These  errors  then  can  be  attributed  to  k 
sec  p  factor  (M  factor)  in  its  calculation  of  MUF.  However,  linear  regression 
did  remove  the  sunset  transition  error.  This  indicates  that  there  is  some 
bias  in  its  critical  frequency  calculation  —  perhaps  in  its  location  of 
control  points. 


BBCONMHD&TIONS 

As  a  result  of  this  study,  the  following  recommendations  are  made: 

1.  The  use  of  MINIMUF-3.5  where  the  only  desired  output  is  MUF  and  FOT 

2.  The  use  of  the  universal  time  ionospheric  data  tape  in  NTP  6  Supp.  1 
predictions 

3.  The  development  of  an  improved  M  factor  equation  for  use  in  MINIMUF- 

3.5 

4.  Augmentation  of  the  MDF  data  base  to  represent  better  the  Atlantic 
Ocean,  northern  latitudes  and  transequatorial  paths 

5.  The  use  of  this  augmented  data  base  to  more  accurately  assess  the 
errors  in  predicted  MUF 

6.  The  assessment  of  minimum  take-off  angle  on  accuracy  of  predicted  MUF 

7.  The  use  of  multiple-linear  regression  to  remove  the  bias  in  predicted 
MUF  due  to  the  bias  in  predicted  fQF2 
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